Once models are created, you can make migrations and database tables from them.
An empty database isn't very interesting, so the next step is to populate it. We used Celery Tasks for that.
Django 2.0 changed how URLs are defined. I ignored that and used the old style re_path because I wasn't really interested in learning the ins and outs of the new `path` function yet.
Here's my first four views
urlpatterns = [
path('api/v1/me/files', v1.file_query),
path('api/v1/login', v1.api_login),
re_path(r'^api/v1/get_file(?P<path>.+)', v1.get_file),
re_path(r'^api/v1/get_thumb(?P<path>.+)', v1.get_thumb),
path('admin/', admin.site.urls),
]The first view is the api endpoint to query your own files.
The second view is an api endpoint to login.
The third and fourth views are to get the contents of a file.
I've skipped over a lot of authentication, authorization and error checking in the views, in order to just focus on the functionality for now. I'm only running this on localhost and trying to get a proof of concept built. I don't like to spend too much time on these sorts of details until I know something is going to be useful or not.
If you've been around Django for a while, you might also notice that I tend not to use Class Based Views. This is just a preference. I think that function based views are easier to read and reason about.
I'll go through the views one by one. You can view the whole file here:
@login_required
def file_query(request):
files = StoredFile.objects.filter(user=request.user)
if request.GET.get('mime_type'):
files = files.filter(mime_type__icontains=request.GET['mime_type'])
if request.GET.get('after'):
after = parse(request.GET['after'])
files = files.filter(Q(start__gte=after) | Q(end__gte=after)).exclude(start__isnull=True)
if request.GET.get('before'):
before = parse(request.GET['before'])
files = files.filter(Q(start__lte=before) | Q(end__lte=before)).exclude(start__isnull=True)
if request.GET.get('sort'):
if request.GET['sort'] == 'newest':
files = files.order_by('-start').exclude(start__isnull=True)
count = 150
if request.GET.get('count'):
count = int(request.GET['count'])
if count > 10000:
count = 10000
if count < 1:
count = 1
paginator = Paginator(files.distinct(), count)
page_files = paginator.get_page(request.GET.get('page', 1))
return JsonResponse({
"results_count": files.count(),
"per_page": count,
"page": page_files.number,
"results": [f.serialize() for f in page_files.object_list]
})
This view starts with an @login_required. You can't query a particular person's files if you don't know who they are.
The next line sets up the base QuerySet for all the user's files.
After that there are three "if" branches that filter the queryset further, by mime_type, sort order, or time.
The next chunk (starting with "count = 150") is about slicing the queryset into pages to enforce that the query/api results are not too big, with a limit of 10000. Ten thousand results is still too many in many cases, we'll figure that out with some testing later.
A Django Paginator is set up, and the results are serialized and returned as a JSON formatted string via JSONResponse.
@csrf_exempt
def api_login(request):
if request.user.is_authenticated:
return JsonResponse(
{"username": request.user.username}
)
creds = json.loads(request.body.decode('utf-8'))
user = authenticate(request, **creds)
if user:
login(request, user)
return JsonResponse(
{"username": user.username}
)
return JsonResponse({"error": "NO"})
This code takes a JSON encoded request body. It decodes the JSON into a python dictionary "creds" and runs it through Django's authentication. This is so a user can log in via the API. There are a _lot_ of caveats to doing authentication this way. This is an incomplete list:
These things are beyond the scope of this post
This, and the following method rely on a feature of nginx that allows protected "internal" redirects. Basically nginx only serves the file if Django says it's OK. This is a really excellent feature and you can read more about it here.
@login_required
def get_file(request, path):
#TODO AuthN and AuthX
file = get_object_or_404(StoredFile, content=path)
response = HttpResponse()
response.status_code = 200
response['Access-Control-Allow-Origin'] = '*'
response['X-Accel-Redirect'] = '/protected{}'.format(file.content.path)
del response['Content-Type']
del response['Content-Disposition']
del response['Accept-Ranges']
del response['Set-Cookie']
del response['Cache-Control']
del response['Expires']
return response
Right now this code assumes that if you have the path to a file, and if you have a valid login on the server, you can get the file. That's only good for my development cases. If there's a second user on the server, this method is one of the first things that will need to change.
The rest of the code sets up the nginx redirect.
@login_required
def get_thumb(request, path):
#TODO AuthN and AuthX
file = get_object_or_404(StoredFile, content=path, mime_type__istartswith="image/jpeg")
hashstring = ".thumbs" + path + "?crop=center&width=200&height=200"
hsh = hashlib.md5()
hsh.update(hashstring.encode('utf-8'))
hsh = hsh.hexdigest()
thumb_path = "/home/{}/.local/share/thumbs/{}.jpg".format(file.user.username, hsh)
if not os.path.isfile(thumb_path):
im = Image.open(file.content.path)
if im.width > im.height: # Landscape
left = int((im.width - im.height) / 2)
im = im.crop((
left,
0,
left + im.height,
im.height))
im = im.resize((200, 200))
else: # Portrait
top = int((im.height - im.width) / 2)
im = im.crop((
0,
top,
im.width,
top + im.width
))
im = im.resize((200, 200))
im.save(thumb_path)
response = HttpResponse()
response.status_code = 200
response['Access-Control-Allow-Origin'] = '*'
response['X-Accel-Redirect'] = '/protected{}'.format(thumb_path)
del response['Content-Type']
del response['Content-Disposition']
del response['Accept-Ranges']
del response['Set-Cookie']
del response['Cache-Control']
del response['Expires']
return response
`get_thumb` is very similar to `get_file` in that it returns a redirect to an internal nginx redirect.
It's different because it accepts an existing file handle and then attempts to create a thumbnail of that image.
This method is meant to expand later, to take arguments for the size of the thumbnail and the crop options. A hash of those options will be taken, and the filename of the thumbnail will be made from the hash. This will avoid re-generating thumbnails on every request. Right now it's assuming center-cropped 200x200 square thumbnails as the only option.
16th June 2018
I won't ever give out your email address. I don't publish comments but if you'd like to write to me then you could use this form.
I'm Issac. I live in Oakland. I make things for fun and money. I use electronics and computers and software. I manage teams and projects top to bottom. I've worked as a consultant, software engineer, hardware designer, artist, technology director and team lead. I do occasional fabrication in wood and plastic and metal. I run a boutique interactive agency with my brother Kasey and a roving cast of experts at Kelly Creative Tech. I was the Director of Technology for Nonchalance during the The Latitude Society project. I was the Lead Web Developer and then Technical Marketing Engineer at Nebula, which made an OpenStack Appliance. I've been building things on the web and in person since leaving Ohio State University's Electrical and Computer engineering program in 2007. Lots of other really dorky things happened to me before that, like dropping out of high school to go to university, getting an Eagle Scout award, and getting 6th in a state-wide algebra competition. I have an affinity for hopscotch.