tag:engineers.sg,2005:/episodes?page=23Engineers.SG2024-03-19T03:04:17Ztag:engineers.sg,2005:Episode/37902019-11-11T16:13:46Z2024-02-29T12:00:56ZBuilding an interactive training environment using JupyterHub - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/xOnCFRuD2Ok" frameborder="0" allowfullscreen></iframe><p>Speaker: Graham Dumpleton, Developer Advocate, OpenShift</p>
<p>JupyterHub is used to run and manage Jupyter notebook instances for multiple users at the same time. Did you know though that you can use JupyterHub to spawn applications other than Jupyter notebooks? Come see how JupyterHub was used to create a multi user interactive training environment. In this talk you will learn what JupyterHub is and how it works. You will be stepped through how JupyterHub was used to implement a multi user workshop environment running in Kubernetes. Instead of creating Jupyter notebooks, JupyterHub was used to create user environments in Kubernetes which gave the user access to an interactive Linux shell environment in their web browser, along with workshop course notes and presenter slides, in the same integrated dashboard view. Because the user environment provided all the tools required for the workshop, users were immediately good to go, with no need to install anything locally on their own computer.</p>
<p>About the speaker:</p>
<p>Graham is the author of mod_wsgi, the Apache module for hosting of Python web applications using the WSGI interface, and the author of wrapt, a decorator and monkey patching library for Python. He also has a keen interest in docker, Kubernetes, and Platform as a Service (PaaS) technologies. He is currently a developer advocate for OpenShift at Red Hat.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37892019-11-11T16:13:23Z2024-03-12T23:01:24ZThe curious case of slow/fast grequests code - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/gsRDYi40MsU" frameborder="0" allowfullscreen></iframe><p>Speaker: Saurabh Hirani, Principal Devops Engineer</p>
<p>If you have made a single HTTP request in Python, you have probably used the requests module. If you have made concurrent HTTP requests in Python, you have probably used the grequests module. I ran into an issue in production which made a grequests based code execute HTTPS GET requests serially without utilising the concurrency goodness promised by grequests. What was more interesting is that depending on the Python modules installed on my system, the same codebase ran fast or slow. Tumbling down that rabbit hole made me understand Python profiling, function tracing, understanding how gevent works and opening a pull request to fix the issue with a gevent based Python module. In this talk, I want to share my learnings with the audience through a working dockerized demo which showcases the following stages for the same codebase while explaining why each stage behaves the way it does: Stage-0 - Trigger which led to the exploration - May run slow/fast for Python 2.7 and slow/fast for Python 3.7 depending on the modules installed on your system. Stage-1 - Baseline with dockerized environment - Runs fast both for Python 2.7 and Python 3.7 Stage-2 - Detection of slowness and verification - Runs slow both for Python 2.7 and Python 3.7 Stage-3 - Python 2.7 fix - Runs fast for Python 2.7 but slow for Python 3.7 Stage-4 - Python 3.7 fix - Patching a gevent based Python module which fixes the issue and makes Python 3.7 fast again.</p>
<p>About the speaker:</p>
<p>Hello - I am Saurabh Hirani - I have been working in the infrastructure automation and tooling domain since 2005. I am currently working as a Principal SRE at Autodesk, Singapore. Python is one of my go-to languages and and I really love the simplicity and power that it yields (pun intended :)). I am giving a talk - "The curios case of slow/fast grequests code" - in which we are going to unravel an interesting concurrency issue I ran into while porting a python 2.7 code to python 3.7 from AWS EC2 to AWS Lambda. We will see how the same codebase runs slow or fast depending on the Python interpreter version and the module environment. In the process we will learn more about Python profiling, tracing and take a peek under the hood of gevent and grequests.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37882019-11-11T16:05:24Z2024-03-18T09:00:46ZDigital process automation (RPA) using TagUI for Python - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/F2aQKWx_EAE" frameborder="0" allowfullscreen></iframe><p>Speaker: Ken Soh, RPA Specialist</p>
<p>TagUI for Python is a Python package for digital process automation (commonly known as RPA). Its simple, expressive and powerful API makes digital process automation fun and easy. Enjoy features such as website automation, visual automation, OCR automation, keyboard and mouse automation in one seamless API. More on this package on its GitHub page - <a href="https://github.com/tebelorg/TagUI-Python">https://github.com/tebelorg/TagUI-Python</a></p>
<p>About the speaker:</p>
<p>Hi! I'm a RPA specialist who created the open-source RPA tool TagUI. My first contact with Python was when building an integration to Python from TagUI. Then I use a lot of Python for my work at Yara Digital Farming. More recently, I created a Python package for digital process automation (RPA) - <a href="https://github.com/tebelorg/TagUI-Python">https://github.com/tebelorg/TagUI-Python</a></p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37872019-11-11T16:05:15Z2024-03-08T02:01:32ZDemystifying RPA with Python - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/YDtKlrFTn2k" frameborder="0" allowfullscreen></iframe><p>Speaker: Thu Ya Kyaw, Developer evangelist</p>
<p>I will talk about what is RPA, how it is being used in the industry, what are the benefits, and how Python can be used to do amazing RPA projects and it's free.</p>
<p>About the speaker:</p>
<p>I learned python back in polytechnic days. It was mainly for things like web scraping back then. Nowadays, everything can be written in Python; artificial intelligence, machine learning, backend services, you name it. I like Python for its simple syntax and coincidentally, I am a fan of automation. It saves time and efforts most of the time, in my opinion. As a machine learning engineer, I mostly use Python and automating tasks with Python is irresistible. For PyCon 2019, I would like to share my experience in automation, mainly RPA use cases, and also about a new Python RPA tool called TagUI.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Thu Ya Kyawtag:engineers.sg,2005:Episode/37862019-11-11T16:05:10Z2024-03-06T00:01:38ZFaster Python apps with open source APM - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/5qrOROkoNOU" frameborder="0" allowfullscreen></iframe><p>Speaker: Aravind Putrevu, Developer Advocate</p>
<p>Slow applications are no fun. Application performance monitoring (APM) makes tracking down issues problems much easier. But which tools should you use? With the release of Elastic APM, there’s a new option. Language server and Python client is fully open source so you can get started with any app. Elastic’s APM was released sometime ago, so it’s a very fresh option for tracing performance problems in Python applications (and other runtimes as well). The basic platform is free so it should be a welcome change for Python developers used to spending huge bills on comparable hosted platforms. It’s useful out of the box for tracing basics on web requests including: - Request details - Response time percentiles - Transaction timelines - Application errors and stack traces - Individual code lines - Distributed Tracing It can also be used to track any custom span in any Python application to find out where time is being lost and users are being slowed down.</p>
<p>About the speaker:</p>
<p>Aravind is passionate about evangelising technology, meeting developers and helping in solving their problems. He is a backend developer and has seven years of development experience. Currently he works at Elastic as Developer Advocate for India & ASEAN and looks after the Developer Relation function of India. Previously, He worked at McAfee Antivirus as a Sr. Software Engineer in Cloud Security Domain. He has deep interest in Search, Machine Learning, Security Incident Analysis and IoT tech. In his free time, he plays around Raspi or a Arduino.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37852019-11-11T15:45:32Z2024-03-19T00:01:36ZBuilding an analytics data pipeline using Airflow and PySpark - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/yYg0GpFKkz4" frameborder="0" allowfullscreen></iframe><p>Speaker: Yohei Onishi, Data Engineer</p>
<p>I have been working on building analytics data pipeline for logistics process in retail industry using Airflow and Spark. I have used Python to author the data pipeline. In this session I will talk about real world use case of airflow including overview of Airflow, how to create your own Airflow cluster, how to integrate Airflow with Spark and how to reduce operation cost using Cloud Composer (managed Airflow cluster service). Note: this talk is based on my talk at PyCon PH 2019 but I will explain more about Airflow and Spark integration.</p>
<p>About the speaker:</p>
<p>Yohei is a data engineer at a global retail company. He has been working on building analytics data pipeline using Apache Airflow recently. He likes OSS and cloud services for data engineering such as Airflow, Spark, Beam and BigQuery</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37842019-11-11T15:45:19Z2024-03-18T18:00:47ZGraylog : Centralized logging of Application Logs - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/jilln5u8T9I" frameborder="0" allowfullscreen></iframe><p>Speaker: Pratibha Jagnere, Senior Python Developer</p>
<p>This talk aims to help developer and sysadmins who want to learn about centralized log management solution. By the end of the talk, Attendees will learn how to use Graylog to visualize application logs, as well as guidance on how to write their own log data extractor for unstructured logs.</p>
<p>About the speaker:</p>
<p>Pratibha is a Senior Python developer over looking the development and maintenance of the backend including APIs and application oriented DevOps work. She is a coder by passion and Freelancer by choice.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37832019-11-11T15:45:11Z2024-02-28T21:00:50ZAutomate the Boring Stuff with Slackbot - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/4r3-5wvi4kA" frameborder="0" allowfullscreen></iframe><p>Speaker: Takanori Suzuki
<br>Vice Chair of PyCon JP Committee, Python Boot Camp</p>
<p>Today, there are many tasks to repeat in the company/community. In addition, we often use chat such as Slack for daily communication. So, I created a chatbot(PyCon JP Bot: <a href="https://github.com/pyconjp/pyconjpbot">https://github.com/pyconjp/pyconjpbot</a>) to automate various boring tasks related to holding PyCon JP. In this talk, I will first explain how to create a chatbot using slackbot (<a href="https://github.com/lins05/slackbot">https://github.com/lins05/slackbot</a>). I will tell you how to registers bot's integration on Slack and how to create a simple bot in Python that responds to specific keywords. And as a specific case, I will explain how to make a bot command to perform the following operations and technical problems. - Emoji reaction - Calculator: SymPy - Karma(plusplus): Peewee - Search issues, display issue details: JIRA API - Create multiple issues from a template: JIRA API, Sheets Spreadsheet API - Search files from Google Drive: Google Drive API - Account management of G Suite(user, alias, group and member): G Suite API - etc. --- notes --- If this talk is accepted, I will participate from Japan. When are you informed of talk adoption? Because Aug 31 is too late to attend from abroad. I presented this talk in PyCon APAC. And I will present in PyCon Thailand, Malaysia and Taiawn. - slide: <a href="https://gitpitch.com/takanory/slides?p=20190224pyconapac">https://gitpitch.com/takanory/slides?p=20190224pyconapac</a> I have a lot of public speaking experience(presenter, lecuturer and MC), but I have experienced in English several times. I have been developing PyConJP Bot since 2016(<a href="https://github.com/pyconjp/pyconjpbot/blob/master/ChangeLog.txt">https://github.com/pyconjp/pyconjpbot/blob/master/ChangeLog.txt</a>). --- outilne --- If you like 30 minutes, I can reduce case studies and adjust the time. The outline of my presentation is below: - Who am I (1m) - Back ground, motivation (2m) - Lots of tasks to hold Conference - Staffs ask me the same thing - Programmer is Lazy - Let's create a secretary - Goal(1m) - You'll learn how to create simple chatbot(one way) - You'll learn how to create interactive bot - You'll learn how to extend bot using libraries and APIs through various case studies - Why Slack? (1m) - Easy to access Slack - I want to do everything in Slack - Simple integration with Incoming Webhook (5m) - System overview(<a href="https://api.slack.com/incoming-webhooks">https://api.slack.com/incoming-webhooks</a>) - Create Incoming Webhooks Integration on Slack: Generate Webhook URL - Send a simple message with cURL - Send a simple message with Requests - Send a complex message with Requests - Summary - How to create slackbot (5m) - System overview - Create bot user on Slack - Install slackbot library - Create a simple bot with slackbot - Simple plugin with slackbot - Run slackbot - Extend slackbot (5m) - listen_to and respond_to decolator - emoji reaction(message.react() method) - Extract parameters on chat message - settings of slackbot - Attachments support - Summary of slackbot - Case study (18m) - Calculator function using SymPy: Install SymPy, sample code of command - Plusplus function using Peewee ORM: Install Peewee, sample code of model and command - Display JIRA issue and Search issues - System overview - Install Python JIRA - Authentication of JIRA API - Get Issue object and search issues code - JIRA API and jira package - Create multiple issues from a template - Motivation - System overview - Google Authorization is very Complex - Get Spreadsheet Data with Sheets API - Create JIRA Issue - Sample template command - Account management of G Suite - Motivation - System overview - Get user list, Insert user - Suspend, Resume, Delete user - I can completely forget Google Admin web site - Summary(1m) - Incoming Webhooks - Slackbot - Slackbot with Libraries and APIs - Next steps (1m) - Let's make your own Slackbot - Let's connect your bot with libraries and APIs - Automate your Boring Stuff with bot - Questions and Answers</p>
<p>About the speaker:</p>
<p>Takanori is a Vice Chairperson of PyCon JP Committee(<a href="http://www.pycon.jp">www.pycon.jp</a>). He is also a director of BeProud Inc.(<a href="http://www.beproud.jp">www.beproud.jp</a>), and his title is "Python Climber". Takanori held PyCon JP 2014 to 2016 as the chairperson. Currently he teaches Python to beginners as a lecturer at Python Boot Camp(pycamp.pycon.jp) all over Japan. In addition, he published several Python books. Tananori plays trumpet, climbs boulder, loves Lego, ferrets and beer.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/38132019-11-11T15:45:04Z2024-03-15T07:00:55ZScaling product support with Python at WhatsApp - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/ZYkYVyPT1PM" frameborder="0" allowfullscreen></iframe><p>Speaker: Mengyi Yuan, Solutions Engineer, Facebook</p>
<p>How do you effectively support a User-Installed Software when the software could be installed in various environments that you have no control of? This talk will share how CLI tools can be built to solve this problem, how the industry has done it over the years, how we did it at WhatsApp with great success, why we chose Python, and inspire the audience to build tools to solve similar real world problems.</p>
<p>About the speaker:</p>
<p>As a Solutions Engineer at Facebook, my job is to help solve business problems by building products. One recent example is a CLI tool built in Python to help scale product support. I am a Python beginner and always learning.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37822019-11-11T15:44:58Z2024-03-18T14:00:56ZAn introduction to Python for Machine Learning with VS Code and Azure - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/aZObMvz87nc" frameborder="0" allowfullscreen></iframe><p>Speaker: Joao Bilhim, Microsoft</p>
<p>Interested on finding how to leverage Python to build and manager machine learning models on Azure? In this session we will start by configuring Visual Studio Code for Python Data Science / Machine Learning and work our way up to building and deploying a Python Machine Learning model.</p>
<p>About the speaker:</p>
<p>Joao is a self-appointed Data Maniac and has been working on applying data to solve complex problems for almost 20 years. Joao is currently passionate about Machine Learning, Python and AI. During the day, Joao leads the Artificial Intelligence and Machine Learning Specialist team for Asia at Microsoft, while, in his spare time, Joao loves to put to work new data techniques and principles to his off-work passions. Joao hails from Portugal and joined Microsoft in 2006.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37812019-11-11T15:36:42Z2023-07-04T21:00:58ZEnding Keynote - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/FXRgtbJrGqo" frameborder="0" allowfullscreen></iframe><p>Speaker: Rich Jones, Founder of Gun.io, Gun.io</p>
<p>About the speaker:</p>
<p>Rich Jones is a prolific open source software developer, archetypal hacker-nomad, founder of the bespoke talent agency Gun.io, and the author of Zappa, Python's serverless framework. Started three years ago, Zappa was the first project which allowed developers to build and deploy complex event-driven web applications to AWS Lambda, allowing for infinite elasticity and massively reduced operations burdens. Zappa is now used in production by thousands of top companies all around the world, helping them save millions of dollars on operations costs. In addition to his open source work, he has provided consulting services for dozens of startups, and was most recently a cloud architect and software developer at the Childhood Cancer Data Lab, where he was responsible for the architecture, application development and operations of a system which processed petabytes of genetic data in order to accelerate the the use of artificial intelligence in the hunt for cures of rare cancers. Rich is known for his fast-paced, wide-ranging and informative talks. He has spoken at DjangoCon, the Chaos Communication Congress, Full Stack Fest and many other conferences across four continents and a dozen countries. In his free time he enjoys skateboarding, trap music and scotch whiskey.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37802019-11-11T15:34:52Z2023-08-10T04:01:51Z2FA, WTF? - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/CWgdwytvdL0" frameborder="0" allowfullscreen></iframe><p>Speaker: Phil Nash, Developer evangelist, Twilio</p>
<p>Everyone is hacking everything. Everything is vulnerable. Your site, your users, even you. Are you worried about this? You should be! Don't worry, I'm not trying to scare you (that much). We have plenty of safeguards against attempts on our applications' user data. We all (hopefully) recognise Two Factor Auth as one of those safeguards, but what actually goes on under the hood of 2FA? We'll take a look into generating one time passwords, implementing 2FA in Python web applications and the only real life compelling use case for QR codes. Together, we'll make the web a more secure place.</p>
<p>About the speaker:</p>
<p>Phil is a developer evangelist for Twilio and a Google Developer Expert. He's been in the web industry for more than 10 years building applications and integrating APIs. He can be found hanging out at meetups and conferences, playing with new technologies, or writing open source code. Phil tweets at @philnash and you can find him elsewhere online at <a href="https://philna.sh">https://philna.sh</a>.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37792019-11-11T15:34:45Z2024-03-14T10:01:08ZMachine Learning Model Development and Operation in DBS - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/NyvYVjyUEew" frameborder="0" allowfullscreen></iframe><p>Speaker: Max Wang, Data Scientist, DBS</p>
<p>The paper will cover 1) business and data problems in DBS, 2) solutions to the problems using machine learning techniques, 3) data platform and tools for model development and deployment in DBS, 4) learnings from our practices.</p>
<p>About the speaker:</p>
<p>I have a PhD in robust optimization from NUS, and after that I continued with my research in Singapore MIT Alliance, Texas A&M University and Singapore Management University from 2008 to 2015. After that I joined SAP Innovation Centre of Singapore as a Data Scientist and started to use Python actively. Now I am the Lead Data Scientist of DBS Singapore Consumer Banking Group and use primarily Python for model development and deployment to solve problems in the banking industry.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37782019-11-11T15:34:39Z2024-02-11T14:01:00ZWhen life gives you Orange, make data speak volumes! - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/QSlNa47J3BI" frameborder="0" allowfullscreen></iframe><p>Speaker: Drishti Jain, Computer Engineer</p>
<p>Abstract Have you ever thought of using data visualization to represent data; but feel that it is a cumbersome process? Worry not – Orange is here to the rescue! Come, dive into the world of this magical open source data mining tool that can also be used as a Python library. Beginner friendly! Description In this talk we will walk through the concept of Data mining and visualization right from the basics. A free and open source tool for data visualization -Orange from a beginner to an advance level will also be covered. We will also cover the concepts of data mining and data warehousing to gain a deeper insight into the concept and help understand the process form the core level. Data mining refers to an autonomous process of discovery of previously unknown patterns which are valid, potentially useful or novel from large database. Orange is Open source machine learning, data visualization and data mining toolkit. Orange features an interactive data visualization platform. The platform provides a wide variety of data analysis and visualization by statistical distributions, scatter plots, box plots, decision trees, heat maps, linear projections to name a few. By carefully selecting attributes in the dataset we can drill down multidimensional data to 2D. It helps in fast prototyping of data analysis workflow. The talk will also cover the recent and diverse existing use cases of Orange like: 1. Analysis of hyperspectra images from the Ryugu asteroid 2. Language support for 50 languages 3. bioinformatics, and many more. By the end of the talk attendees would have a clear knowledge of • What Orange is • How they can use it in work or side projects • Be able to use it as a separate tool or embed it as a library in their existing Python projects And use all functionalities of Orange of data mining and visualization to the fullest!</p>
<p>About the speaker:</p>
<p>Drishti is a Computer Engineer at heart and a technology enthusiast. She loves to use technology to help the less fortunate. She believes in democratizing opportunities and brings knowledge of the latest developments in the fast-moving field of technology to deserving students, and keep them up to date and well-equipped for their respective professional careers. She has spoken at conferences across the globe and is also a social entrepreneur. Her non-profit organisation - Samyak Drishti Foundation works in environment, education and healthcare sectors and operates in 10 cities across India. In her spare time, she likes to paint nature, explore new places and anchor live shows</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37772019-11-11T15:29:41Z2024-02-20T21:01:00ZHow to train a wacky language model - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/uvUOjJrv40g" frameborder="0" allowfullscreen></iframe><p>Speaker: Jonathan Heng, Software Developer</p>
<p>I will talk about GPT-2, a language model developed by Open AI, and how we can finetune it with various training sources to get interesting results. These will be demonstrated in Tensorflow. Ethical implications of such models will be discussed as well. If time permits, I will also demonstrate a simple deployment as a flask app on a cloud platform (GCP).</p>
<p>About the speaker:</p>
<p>Jonathan is a software developer at ThoughtWorks. He has 3 years of experience in building machine learning models for both research and businesses. While waiting for his model to train, Jonathan listens to music and contemplates when the bots will rule the world.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37762019-11-11T15:29:35Z2023-09-19T16:01:38ZDay 2 Opening Keynote - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/u0U7WnCq9xs" frameborder="0" allowfullscreen></iframe><p>Speaker: Wan Ting Poh, Director Data Science, Allianz</p>
<p>About the speaker:</p>
<p>Wan Ting is Allianz’s Director Data Science for Asia Pacific. She is responsible for leading and developing a team of data scientists in the region, with the primary aim to drive innovation, deliver end-to-end data science services, and continue the digitalisation of the Allianz business. Wan Ting also holds the parallel role of Chief Data Scientist at FinOS Technology, a wholly-owned subsidiary of Allianz, which is focused on increasing access to financial services for the underbanked and underinsured. Between 2017 and 2019, Wan Ting held the office of Managing Director of Girls in Tech Singapore – a non-profit organization that empowers, educates and engages girls and women who are passionate in technology. Wan Ting’s leadership expanded the organization’s community, positively impacted more than 700 participants, and substantially increased working capital for the group – efforts which ensures the Singapore chapter remains relevant, viable and sustainable. A strong believer that technology can drive the world to be a better place, Wan Ting recognizes the importance of diversity in the digital age and strives towards the day where everyone is empowered in STEM. Wan Ting holds a Master degree in statistics and a Bachelor degree in Computational Biology, both from the National University of Singapore.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37752019-11-11T15:29:28Z2024-02-23T16:01:30ZPython on AWS Lambda - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/o40R_qb4AyQ" frameborder="0" allowfullscreen></iframe><p>Speaker: Keith Rozario, IT Architect</p>
<p>"Serverless functions like AWS lambda are the new paradigm for developers. In this talk I want to cover a few advance topics when dealing with serverless functions on AWS lambda, specifically when it relates to python. A large chunk of documentation on AWS lambda is usually written for JavaScript, I wanted to give a talk to cover specifically for python devs. - Intro to Lambda functions on AWS. What they are, how they work, and how Python based lambda functions rock! - Using AWS Lambda Layers to package re-usable code, and even for storing python packages. This simplifies deployments, reduces storage limits, and even adds to better traceability to functions. - Using io and fileobjs to process data in memory of the lambda, rather than writing files to disk. This not just improves performance, but overcomes the hard 512MB limit on file storage -- since we can get up to 3GB of memory. All data into and out of the lambda function is via the network interface, saving to disk should be a last resort since everything written to disk needs to be read out into memory before shipping out through the network! - How to multi-thread within a lambda function -- ""Due to the Lambda execution environment not having /dev/shm (shared memory for processes) support, you can’t use multiprocessing.Queue or multiprocessing.Pool"" -- hence we have to fallback to Pipe, but it does help improve performance of the functions overall. - Place code outside the handler method to have it run on 'cold-start' and persist that data throughout multiple invocations -- but remember to delete files on disk you no longer need since they also persist. - We can then cover the serverless framework to give each individual lambda a very specific IAM role, and a use layers to give them specific dependencies, turning lambda functions into 'micro-service' like applications. Best Practices when using the framework for Python including using environment variables, - If timing permits, we can also delve into building our own layers using two methods, first the common method of using a Docker Container (lambci), second, an idea I've been experimenting on, which is using Lambda themselves to package lambda layers !</p>
<p>About the speaker:</p>
<p>Keith is an architect by day, and serverless enthusiast by night. He currently maintains Klayers, a project aimed at providing the most popular python packages as publicly available AWS lambda layers.</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SGtag:engineers.sg,2005:Episode/37742019-11-11T15:29:22Z2024-03-12T12:01:34ZDemystifying Time Series Forecasting using Python - PyCon SG 2019<iframe width="560" height="315" src="https://www.youtube.com/embed/qhSEmJ6v6BA" frameborder="0" allowfullscreen></iframe><p>Speaker: Vaibhav Srivastav, Data Scientist</p>
<p>Nowadays, it is hard to find a company that doesn’t collect various time-dependent data in different forms, for instance, it can be a daily number of visitors and monthly sales for online stores, available resources and stock for factories, number of food poisoning cases for hospitals, and so on. And the reason why all that data is carefully collected, because it can provide meaningful insides not only about the past but can be used to predict and prepare for the future. In this presentation, we discuss how to analyze and forecast those data, that is called time series. Many people already did that many times while trying to predict the weather on the weekend, guessing the currency exchange rate for tomorrow, or just by expecting great discounts on Christmas sales. Of course, some patterns are truly obvious, like weekly or monthly changes, and overall tendency, others are not. However, all these aspects can be formalized and learned automatically using the power of mathematics and computer science. The first part is dedicated to the theoretical introduction of time series, where listeners can learn or refresh in memory the essential aspects of time series’ representations, modeling, and forecasting. In the second part, we dive into the most popular time series forecast models - stochastic models (e.g., Autoregressive integrated moving average (ARIMA)), artificial neural networks (e.g., seasonal recurrent neural network) and Support Vector Machines (SVR). Along the way, we show at practice how these models can be applied to a real-world dataset of Singapore visits by providing examples using such popular Python libraries as StatsModels, Prophet, scikit-learn, and keras. With these guidelines in mind, you should be better equipped to deal with time series in your everyday work and opt-in for the right tools to analyze them. To follow the talk it’s not required any prior knowledge of time series analysis, but the basic understanding of mathematics and machine learning approaches could be quite helpful.</p>
<p>About the speaker:</p>
<p>Hi! I am a Data Scientist working with Deloitte Consulting LLP, I work with Fortune Technology 10 clients to help them make data-driven (read profitable) decisions. Prior to this I have worked with startups across India to build Social Media Analytics Dashboards, Chatbots, Recommendation Engines and Forecasting Models. My core interest lie in Natural Language Processing, Machine Learning/ Statistics and Product development. In my free time I give talks and participate in local PyData/ PyUserGroup meetups, have previously given a talk at Gartner Data and Analytics Summit, PyCon India, PyCon APAC (Philippines), PyCon Korea, PyCon Malaysia and Google Cloud Summit! If Data is what floats your boat, then coffee is on me! :D</p>
<p>Event Page: <a href="https://pycon.sg/">https://pycon.sg/</a></p>
<p>Produced by Engineers.SG</p>Engineers.SG