PyCon APAC 2022｜一般演講 Talks｜國泰金控 Cathay Financial Holdings / 美光科技 Micron 冠名贊助 ✏️ 共筆 Note：https://hackmd.io/@pycontw/rkh-wTXyi ?? Slido：https://app.sli.do/event/jfzDeYQ9auhfLV1U9uRRBN ? 投影片 Slides：https://github.com/ShreyaKhurana/PyconAPAC22 ? 語言 Language：英文 English ? 層級 Level：中階 Intermediate ? 分類 Category：自然語言處理 Natural Language Processing ? 摘要 Abstract ? From Googling for the reviews of that new deli place to watching Parasite with subtitles, sequence-to-sequence learning is behind a variety of applications - machine translation, speech recognition, chatbots etc. In this talk, I share my experiences with modeling real-world messy data with attention-based transformer and CNN models - the good, the bad, and the ugly. I'll discuss some of the lessons that we learned through experimentation on sequence-to-sequence models - about architecture size, vocabulary, the relation between validation and training error, etc. Working in ML makes you realize one thing–it’s not always black and white and it doesn’t have to be a black box! The process of building good NLP models is not a straight line. Not-so-little decisions can often require tons of experiments, but in this talk, I share my modeling experience and lessons learned so your walk isn't as random as mine was! ? 說明 Description ? I’ll start by discussing briefly how sequence (input) to sequence (output) deep learning architectures have evolved from RNNs to recent advancements in Transformer, BERT and GPT3. I’ll talk about the different applications of sequence to sequence learning, which will help the audience understand if it’s a good fit for their data. I'll discuss some results of hyperaparameter and data tuning that have been shown in the literature and are often used as starting points during modeling. Data collection, data cleaning and pre-processing play a significant role in effectively solving any data science problems. Particularly, subwords and byte-pair processing have been immensely helpful in generalizing the task for rare words. FastBPE[1] and subword NMT[2] are two tools that are widely used for this purpose. I’ll show how small errors during subword preprocessing can lead to very different outputs and harm the prediction quality. I’ll also address the common questions we face as ML practitioners about the architecture, hyperparameters and training metrics. While research literature points to clear patterns of performance with data size and number of model parameters, real data is often messy and has its own set of nuances. I'll be sharing some lessons and tricks to help make practical modeling choices. Sometimes, having enough labelled data is hard, and annotation is difficult. In these cases, models need to be trained with small datasets, which are often less representative. We discuss what approaches are viable in these cases. This talk will help get NLP practitioners an understanding of some common modeling pitfalls and hopefully help develop a sense of what works better in practice. [1] https://github.com/glample/fastBPE [2] https://github.com/rsennrich/subword-nmt ? 講者介紹 About Speaker - Shreya Khurana ? Hi, I'm a data scientist in the Domain Search team at GoDaddy which researches, develops, and deploys deep learning models for Godaddy’s domain business. I enjoy working with data in general and more specifically building NLP models. I'm a Python enthusiast and enjoy sharing my learnings with the community - I've previously presented at the Grace Hopper Conference, PyCon US, EuroPython and GeoPython. #pycontw #pyconapac2022 #python #sequencetosequence #nlp #machinelearning Follow “PyCon Taiwan” ⭐️ Official Website: https://tw.pycon.org ⭐️ Facebook: https://www.facebook.com/pycontw ⭐️ Instagram: https://www.instagram.com/pycontw ⭐️ Twitter: https://twitter.com/PyConTW ⭐️ LinkedIn: https://www.linkedin.com/company/pycontw ⭐️ Blogger: https://pycontw.blogspot.com ... https://www.youtube.com/watch?v=vI74uLKk4ag

Day 1, R1 10:50‑11:20 Donkey Car 是 Adam Conway 在 2017 年發起的專案，一開始是想做一個縮小版的自走車，因此使用 1/10 比例的 RC 車作為車體，並用 OpenCV 裡的車道跟隨技術，但他的夥伴 Will Roscoe 希望能像 Google 和 Tesla 一樣可以使用深度學習技術打造自動駕駛系統，因此很快的 Donkey Car 就加入了神經網路模型。 Donkey Car 是一個開源機器學習的自走車專案，車上唯一的感測器就是相機。藉由操作 Donkey Car，我們可以將機器學習的過程跑一遍，包括訓練資料蒐集與處理、模型選擇與調整、實機訓練、測試驗證等。從無到有瞭解機器學習的原理，並操作整個過程，最後透過車輛的自走看到實際的成果。而最有趣的地方不只是機器學習的部份，我們可以自由的改裝車上的硬體或是修改車道的設計比較各項變化，這都能讓我們離真實的自駕車更靠近。 Donkey Car 使用的硬體包括 Raspberry Pi、RC Car(不限)、馬達控制板和最重要的魚眼相機等。使用的軟體包括 Raspbian 系統加上 Keras/Tensorflow 神經網路，搭配 Tornado 做的 web control。 我們將分享專案的技術細節、實做上會遇到的問題與相關學習資源，希望有更多人能一起來當車友。 Slides: https://www.slideshare.net/raspberrypi-tw/donkey-car-raspberry-pi Speaker: sosorry Hi, I'm sosorry. ... https://www.youtube.com/watch?v=D121LQqLdi4