Speaker: Keith Rozario, IT Architect

"Serverless functions like AWS lambda are the new paradigm for developers. In this talk I want to cover a few advance topics when dealing with serverless functions on AWS lambda, specifically when it relates to python. A large chunk of documentation on AWS lambda is usually written for JavaScript, I wanted to give a talk to cover specifically for python devs. - Intro to Lambda functions on AWS. What they are, how they work, and how Python based lambda functions rock! - Using AWS Lambda Layers to package re-usable code, and even for storing python packages. This simplifies deployments, reduces storage limits, and even adds to better traceability to functions. - Using io and fileobjs to process data in memory of the lambda, rather than writing files to disk. This not just improves performance, but overcomes the hard 512MB limit on file storage -- since we can get up to 3GB of memory. All data into and out of the lambda function is via the network interface, saving to disk should be a last resort since everything written to disk needs to be read out into memory before shipping out through the network! - How to multi-thread within a lambda function -- ""Due to the Lambda execution environment not having /dev/shm (shared memory for processes) support, you can’t use multiprocessing.Queue or multiprocessing.Pool"" -- hence we have to fallback to Pipe, but it does help improve performance of the functions overall. - Place code outside the handler method to have it run on 'cold-start' and persist that data throughout multiple invocations -- but remember to delete files on disk you no longer need since they also persist. - We can then cover the serverless framework to give each individual lambda a very specific IAM role, and a use layers to give them specific dependencies, turning lambda functions into 'micro-service' like applications. Best Practices when using the framework for Python including using environment variables, - If timing permits, we can also delve into building our own layers using two methods, first the common method of using a Docker Container (lambci), second, an idea I've been experimenting on, which is using Lambda themselves to package lambda layers !

About the speaker:

Keith is an architect by day, and serverless enthusiast by night. He currently maintains Klayers, a project aimed at providing the most popular python packages as publicly available AWS lambda layers.

Event Page: https://pycon.sg/

Produced by Engineers.SG