Published By
Created On
8 May 2021 10:21:39 UTC
Transaction ID
Cost
Safe for Work
Free
Yes
Involution: Inverting the Inherence of Convolution for Visual Recognition (Research Paper Explained)
#involution #computervision #attention
Convolutional Neural Networks (CNNs) have dominated computer vision for almost a decade by applying two fundamental principles: Spatial agnosticism and channel-specific computations. Involution aims to invert these principles and presents a spatial-specific computation, which is also channel-agnostic. The resulting Involution Operator and RedNet architecture are a compromise between classic Convolutions and the newer Local Self-Attention architectures and perform favorably in terms of computation accuracy tradeoff when compared to either.
OUTLINE:
0:00 - Intro & Overview
3:00 - Principles of Convolution
10:50 - Towards spatial-specific computations
17:00 - The Involution Operator
20:00 - Comparison to Self-Attention
25:15 - Experimental Results
30:30 - Comments & Conclusion
Paper: https://arxiv.org/abs/2103.06255
Code: https://github.com/d-li14/involution
Abstract:
Convolution has been the core ingredient of modern neural networks, triggering the surge of deep learning in vision. In this work, we rethink the inherent principles of standard convolution for vision tasks, specifically spatial-agnostic and channel-specific. Instead, we present a novel atomic operation for deep neural networks by inverting the aforementioned design principles of convolution, coined as involution. We additionally demystify the recent popular self-attention operator and subsume it into our involution family as an over-complicated instantiation. The proposed involution operator could be leveraged as fundamental bricks to build the new generation of neural networks for visual recognition, powering different deep learning models on several prevalent benchmarks, including ImageNet classification, COCO detection and segmentation, together with Cityscapes segmentation. Our involution-based models improve the performance of convolutional baselines using ResNet-50 by up to 1.6% top-1 accuracy, 2.5% and 2.4% bounding box AP, and 4.7% mean IoU absolutely while compressing the computational cost to 66%, 65%, 72%, and 57% on the above benchmarks, respectively. Code and pre-trained models for all the tasks are available at this https URL.
Authors: Duo Li, Jie Hu, Changhu Wang, Xiangtai Li, Qi She, Lei Zhu, Tong Zhang, Qifeng Chen
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/
BiliBili: https://space.bilibili.com/1824646584
If you want to support me, the best thing to do is to share out the
...
https://www.youtube.com/watch?v=pH2jZun8MoY
Convolutional Neural Networks (CNNs) have dominated computer vision for almost a decade by applying two fundamental principles: Spatial agnosticism and channel-specific computations. Involution aims to invert these principles and presents a spatial-specific computation, which is also channel-agnostic. The resulting Involution Operator and RedNet architecture are a compromise between classic Convolutions and the newer Local Self-Attention architectures and perform favorably in terms of computation accuracy tradeoff when compared to either.
OUTLINE:
0:00 - Intro & Overview
3:00 - Principles of Convolution
10:50 - Towards spatial-specific computations
17:00 - The Involution Operator
20:00 - Comparison to Self-Attention
25:15 - Experimental Results
30:30 - Comments & Conclusion
Paper: https://arxiv.org/abs/2103.06255
Code: https://github.com/d-li14/involution
Abstract:
Convolution has been the core ingredient of modern neural networks, triggering the surge of deep learning in vision. In this work, we rethink the inherent principles of standard convolution for vision tasks, specifically spatial-agnostic and channel-specific. Instead, we present a novel atomic operation for deep neural networks by inverting the aforementioned design principles of convolution, coined as involution. We additionally demystify the recent popular self-attention operator and subsume it into our involution family as an over-complicated instantiation. The proposed involution operator could be leveraged as fundamental bricks to build the new generation of neural networks for visual recognition, powering different deep learning models on several prevalent benchmarks, including ImageNet classification, COCO detection and segmentation, together with Cityscapes segmentation. Our involution-based models improve the performance of convolutional baselines using ResNet-50 by up to 1.6% top-1 accuracy, 2.5% and 2.4% bounding box AP, and 4.7% mean IoU absolutely while compressing the computational cost to 66%, 65%, 72%, and 57% on the above benchmarks, respectively. Code and pre-trained models for all the tasks are available at this https URL.
Authors: Duo Li, Jie Hu, Changhu Wang, Xiangtai Li, Qi She, Lei Zhu, Tong Zhang, Qifeng Chen
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/
BiliBili: https://space.bilibili.com/1824646584
If you want to support me, the best thing to do is to share out the
...
https://www.youtube.com/watch?v=pH2jZun8MoY
Author
Content Type
Unspecified
video/mp4
Language
English
More from the publisher
Controlling
VIDEO
[NEWS
news-openai-model-generates-python-2
This code completion engine can write an entire function from just the name! OpenAI demonstrates what happens when you learn a language model on thousands of GitHub Python repositories.
Source Clip: https://youtu.be/fZSFNUT6iY8
Full Video: https://www.pscp.tv/Microsoft/1OyKAYWPRrWKb
Kite: https://kite.com/
TabNine: https://www.tabnine.com/
Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
...
https://www.youtube.com/watch?v=utuz7wBGjKM
Transaction
Created
1 year ago
Content Type
Language
video/mp4
English
Controlling
VIDEO
GREG
greg-sam-are-back!-(+-q-star-is-agi)
#openai #qstar #samaltman
Sam Altman and Greg Brockman return to OpenAI with a new board of directors. We also learn that internally, a letter touting an algorithm called Q* (Q-Star) was cited as the reason for the board's big reaction. Has OpenAI invented AGI?
References:
https://twitter.com/MichaelTrazzi/status/1727473723597353386?t=WwZ63LrdeSQCZil5dAr-fA&s=09
https://twitter.com/eshear/status/1727342399297630393?t=kojWvCZyacaVS15Hp6lgmQ&s=09
https://twitter.com/JacquesThibs/status/1727134087176204410
https://twitter.com/knowclarified/status/1727126082334416957
https://twitter.com/NathanLands/status/1727144966865195479
https://twitter.com/sama/status/1727206691262099616
https://twitter.com/gdb/status/1727230819226583113
https://twitter.com/karaswisher/status/1727154210599534806
https://twitter.com/OpenAI/status/1727206187077370115?t=7QDuCvt4D7PbkPkoPNWHpQ&s=09
https://twitter.com/sama/status/1727207458324848883?t=AhBZkbUpnzSZB0pvQ6jDuQ&s=09
https://twitter.com/karaswisher/status/1727071629556293935?t=GaAs0IIHZmSSZCer7NY0uA&s=09
https://twitter.com/karaswisher/status/1726780943090827736?t=Q-m6QxZk9UiS8Gn9AOxcfQ&s=09
https://twitter.com/_akhaliq/status/1726677052395344064?t=94kYx08Efh9QCnHe_51A1w&s=09
https://twitter.com/TechCrunch/status/1726684075996623284
https://twitter.com/ilyasut/status/1726590052392956028
https://twitter.com/DrJimFan/status/1726514885578854790
https://twitter.com/E0M/status/1726674821717921927
https://twitter.com/terrorproforma/status/1726530506412716537
https://twitter.com/marktenenholtz/status/1726585324271481332
https://twitter.com/BorisMPower/status/1726600452664647863
https://twitter.com/Benioff/status/1726695914105090498
https://twitter.com/alexeheath/status/1726695001466585231
https://twitter.com/satyanadella/status/1726509045803336122
https://twitter.com/karaswisher/status/1726479707104035092?t=VJgTkvPms3T3YeNM0uIQPg&s=09
https://twitter.com/karaswisher/status/1726477828072382480?t=VJgTkvPms3T3YeNM0uIQPg&s=09
https://en.wikipedia.org/wiki/Emmett_Shear
https://twitter.com/_akhaliq/status/1726467905527611441?t=VJgTkvPms3T3YeNM0uIQPg&s=09
https://twitter.com/ashleevance/status/1726469283734274338?t=VJgTkvPms3T3YeNM0uIQPg&s=09
https://twitter.com/emilychangtv/status/1726457543629914389?t=8T1jtAcSkTt512atrTcJ0g&s=09
https://twitter.com/rowancheung/status/1726342477874102604
Memes:
https://twitter.com/nikitabier/status/1725711315769917691
https://twitter.com/declangessel_/status/1725937251857948755?t=Ltpf31mrCrTOo2F73NgTgw&s=09
https://twitter.com/smylkshmn/status/1725748709953274092
https://twitter.com/emmalrichter/status/1725890869029388418
https://twitter.com/abacaj/status/1726041580140744729
https://twitter.com/luke_metro/status/1726117222480027734
https://twitter.com/sperosck/status/1726028948545339774
https://twitter.com/ConfusingImages/status/1725940641257934944?t=Ltpf31mrCrTOo2F73NgTgw&s=09
https://twitter.com/DeadCaitBounce/status/1726016523565781152?t=Ltpf31mrCrTOo2F73NgTgw&s=09
https://twitter.com/InternetH0F/status/1726006255473262730?t=32kOq4KoJuY_zWTflkFwbA&s=09
https://twitter.com/marktenenholtz/status/1725715287176061041?t=Z-nKJceZ-1EB2wvla0tdQg&s=09
https://twitter.com/satyanutella_/status/1725675331565093151
https://twitter.com/netcapgirl/status/1725877676869578788
https://twitter.com/grokcamp/status/1725712607481352428
https://twitter.com/netcapgirl/status/1725872853092827314
https://twitter.com/theojaffee/status/1725641656320532500
https://twitter.com/netcapgirl/status/1725881257605620190
https://twitter.com/jachaseyoung/status/1725825888057753614
https://twitter.com/a_musingcat/status/1725865515975913810
https://twitter.com/growing_daniel/status/1725731742781497732
Links:
Homepage: https://ykilcher.com
Merch: https://ykilcher.com/merch
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
LinkedIn: https://www.linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
...
https://www.youtube.com/watch?v=GFg98TDqCpw
Transaction
Created
5 months ago
Content Type
Language
video/mp4
English
Controlling
VIDEO
CELEB
celebrating-100k-subscribers!-(w-channel
#yannickilcher #machinelearning #100k
OUTLINE:
0:00 - 100k!
1:00 - Announcements & Thanks
3:55 - Channel Statistics
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/
BiliBili: https://space.bilibili.com/1824646584
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
...
https://www.youtube.com/watch?v=ifBI2jTaAEo
Transaction
Created
1 year ago
Content Type
Language
video/mp4
English
Controlling
VIDEO
ML NE
ml-news-live!-(dec-30,-2021)-anonymous
#mlnews #live #tensorflow
The greatest news, now fully live!
Sponsor: Weights & Biases
https://wandb.me/yannic
References:
https://github.com/tensorflow/tensorflow/issues/53549
https://www.reddit.com/r/ProgrammerHumor/comments/rpkxg2/an_honest_guy/
https://www.scmp.com/news/china/science/article/3160997/chinese-scientists-develop-ai-prosecutor-can-press-its-own?utm_source=pocket_mylist
https://twitter.com/minimaxir/status/1470913487085785089?ref_src=twsrc%5Etfw
https://www.gibsondunn.com/new-york-city-enacts-law-restricting-use-of-artificial-intelligence-in-employment-decisions/
https://twitter.com/klivdahl/status/1475220450598924297
https://www.marketwatch.com/story/this-fund-driven-by-artificial-intelligence-is-ditching-big-tech-heres-what-its-doing-instead-11640778192
https://www.aicrowd.com/challenges/food-recognition-benchmark-2022?utm_source=pocket_mylist
https://twitter.com/abidlabs/status/1473821989517283330
https://www.gradio.app/
https://github.com/activeloopai/Hub?utm_source=pocket_mylist
https://github.com/jalammar/ecco
https://github.com/Continvvm/continuum
https://www.reddit.com/r/MachineLearning/comments/rr17f9/p_45_times_faster_hugging_face_transformer/?utm_source=pocket_mylist
https://github.com/ELS-RD/transformer-deploy
https://github.com/facebookresearch/SLIP
https://arxiv.org/abs/2112.12750
https://www.youtube.com/watch?v=CaS-ruEiUcg
https://github.com/joeljang/music2video
https://github.com/chandar-lab/RLHive
https://rlhive.readthedocs.io/en/stable/quickstart.html
https://github.com/KomputeProject/kompute
https://github.com/Rishit-dagli/CPPE-Dataset
https://www.reddit.com/r/MachineLearning/comments/rr17f9/p_45_times_faster_hugging_face_transformer/
https://github.com/ELS-RD/transformer-deploy
https://mobile.twitter.com/cairuia/status/1472931906060754948
https://istm.no/
https://en.wikipedia.org/wiki/Tsetlin_machine
https://www.reddit.com/r/MachineLearning/comments/ro2567/r_jojogan_one_shot_face_stylization/
https://github.com/mchong6/JoJoGAN
https://www.theguardian.com/commentisfree/2021/dec/25/worried-about-super-intelligent-machines-they-are-already-here
https://www.theguardian.com/commentisfree/2021/oct/02/the-truth-about-artificial-intelligence-it-isnt-that-honest
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https:
...
https://www.youtube.com/watch?v=oRkhACMRy7Y
Transaction
Created
1 year ago
Content Type
Language
video/mp4
English
Controlling
VIDEO
[ML N
ml-news-new-imagenet-sota-uber's-h3
#truthfulqa #efficientnet #laion400M
Your regularly irregular updates on what's happening in the Machine Learning world.
OUTLINE:
0:00 - Intro
0:20 - TruthfulQA benchmark shines new light on GPT-3
2:00 - LAION-400M image-text-pair dataset
4:10 - GoogleAI's EfficientNetV2 and CoAtNet
6:15 - Uber's H3: A hexagonal coordinate system
7:40 - AWS NeurIPS 2021 DeepRacer Challenge
8:15 - Helpful Libraries
9:20 - State of PyTorch in September 2021
10:05 - Physics-Based Deep Learning Book
10:35 - Music-conditioned 3D dance generation
11:40 - Stallman's take on legal issues with Codex
12:20 - Tensorflow DirectML on AMD GPUs
13:00 - Schmidhuber Blog: Turing Oversold
References:
TruthfulQA - A benchmark assessing truthfulness of language models
https://owainevans.github.io/pdfs/truthfulQA_lin_evans.pdf
LAION-400M image-text-pair dataset
https://laion.ai/laion-400-open-dataset/
https://laion.ai/#top
https://gogetfunding.com/help-us-build-the-worlds-largest-open-billion-scale-image-text-dataset-perfect-for-training-dall-e-clip-other-multimodal-models/
https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fsplunk.vra.ro&index=laion_400m_128G&query=yellow+train
GooleAI releases EfficientNetV2 and CoAtNet
https://ai.googleblog.com/2021/09/toward-fast-and-accurate-neural.html
Uber's H3 hexagonal coordinate systems
https://eng.uber.com/h3/?utm_source=pocket_mylist
NeurIPS 2021 DeepRacer Challenge
https://www.aicrowd.com/challenges/neurips-2021-aws-deepracer-ai-driving-olympics-challenge?utm_source=pocket_mylist
https://aws.amazon.com/deepracer/
https://gitlab.aicrowd.com/deepracer/neurips-2021-aws-deepracer-starter-kit/-/tree/master/deepracer-gym
Helpful Libraries
https://github.com/rom1504/img2dataset
https://github.com/facebookresearch/vissl?utm_source=pocket_mylist
https://github.com/pyg-team/pytorch_geometric
https://aws.amazon.com/blogs/machine-learning/announcing-the-amazon-s3-plugin-for-pytorch/
State of PyTorch in September 2021
https://dev-discuss.pytorch.org/t/state-of-pytorch-core-september-2021-edition/332
Physics-Based Deep Learning Book
http://physicsbaseddeeplearning.org/intro.html
https://arxiv.org/pdf/2109.05237.pdf
Music Conditioned 3D dance generation
https://ai.googleblog.com/2021/09/music-conditioned-3d-dance-generation.html
Richard Stallman on Codex legal issues
https://news.slashdot.org/story/21/09/18/0432224/richard-stallman-shares-his-concerns-about-githubs-copilot----and-about-github
Tensorflow DirectML on AMD
https://wccftech.com/amd-microsoft-bring-tensorflow-directml-to-life-4x-improvement-with-rdna-2-gpus/
Schmidhuber: Turing Oversold
https://people.idsia.ch//~juergen/turing-oversold.html
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: ht
...
https://www.youtube.com/watch?v=DkojaN7_f4E
Transaction
Created
1 year ago
Content Type
Language
video/mp4
Controlling
VIDEO
BLEUR
bleurt-learning-robust-metrics-for-text
Proper evaluation of text generation models, such as machine translation systems, requires expensive and slow human assessment. As these models have gotten better in previous years, proxy-scores, like BLEU, are becoming less and less useful. This paper proposes to learn a proxy score and demonstrates that it correlates well with human raters, even as the data distribution shifts.
OUTLINE:
0:00 - Intro & High-Level Overview
1:00 - The Problem with Evaluating Machine Translation
5:10 - Task Evaluation as a Learning Problem
10:45 - Naive Fine-Tuning BERT
13:25 - Pre-Training on Synthetic Data
16:50 - Generating the Synthetic Data
18:30 - Priming via Auxiliary Tasks
23:35 - Experiments & Distribution Shifts
27:00 - Concerns & Conclusion
Paper: https://arxiv.org/abs/2004.04696
Code: https://github.com/google-research/bleurt
Abstract:
Text generation has made significant advances in the last few years. Yet, evaluation metrics have lagged behind, as the most popular choices (e.g., BLEU and ROUGE) may correlate poorly with human judgments. We propose BLEURT, a learned evaluation metric based on BERT that can model human judgments with a few thousand possibly biased training examples. A key aspect of our approach is a novel pre-training scheme that uses millions of synthetic examples to help the model generalize. BLEURT provides state-of-the-art results on the last three years of the WMT Metrics shared task and the WebNLG Competition dataset. In contrast to a vanilla BERT-based approach, it yields superior results even when the training data is scarce and out-of-distribution.
Abstract: Thibault Sellam, Dipanjan Das, Ankur P. Parikh
Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
...
https://www.youtube.com/watch?v=rl4nUngiR2k
Transaction
Created
1 year ago
Content Type
Language
video/mp4
English
Controlling
VIDEO
[ML N
ml-news-nvidia-gtc'21-deepmind-buys
#gtc21 #mlnews #mujoco
Register to GTC'21 and Win a RTX 3090: https://nvda.ws/2Y2B5ni
OUTLINE:
0:00 - Intro
0:15 - Sponsor: NVIDIA GTC'21
5:35 - DeepMind buys & Open-Sources MuJoCo
7:25 - PyTorch 1.10 Released
9:10 - Google Predicts Spreadsheet Formulas
11:25 - handtracking.io
12:25 - Cell Instance Segmentation Challenge
13:00 - Helpful Libraries
17:50 - Waymo cars keep turning into same dead-end
19:35 - BlueRiver balances tractors
References:
DeepMind buys & open-sources MuJoCo
https://deepmind.com/blog/announcements/mujoco
PyTorch 1.10 released
https://pytorch.org/blog/pytorch-1.10-released/
https://developer.nvidia.com/blog/cuda-graphs/
GoogleAI predicts spreadsheet formulas
https://ai.googleblog.com/2021/10/predicting-spreadsheet-formulas-from.html
Handtracking in Browser
https://handtracking.io/
https://handtracking.io/draw_demo/
Sartorius Cell Instance Segmentation Competition
https://www.kaggle.com/c/sartorius-cell-instance-segmentation/
Helpful Libraries
https://github.com/IntelLabs/control-flag
https://github.com/facebookresearch/salina
https://github.com/facebookresearch/salina/tree/main/salina_examples/rl/a2c/mono_cpu
https://github.com/ydataai/ydata-synthetic
https://syntheticdata.community/
https://github.com/ydataai/ydata-synthetic/blob/master/examples/regular/gan_example.ipynb
https://medium.com/aimstack/aim-3-0-0-the-foundations-for-open-source-open-metadata-ml-platform-f3969755d55
https://github.com/aimhubio/aim
https://robustbench.github.io/
Waymo cars keep coming to same dead-end over and over
https://sanfrancisco.cbslocal.com/2021/10/14/dead-end-sf-street-plagued-with-confused-waymo-cars-trying-to-turn-around-every-5-minutes/
BlueRiver balances tractors
https://www.linkedin.com/posts/lredden_blue-river-is-building-the-boston-dynamics-activity-6850873662959169536-8sue/
https://bluerivertechnology.com/ourmethods/
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/1824646584
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL
...
https://www.youtube.com/watch?v=U8Rmfb8aZXE
Transaction
Created
1 year ago
Content Type
Language
video/mp4
English
Controlling
VIDEO
ACCEL
accelerating-deep-learning-by-focusing
What if you could reduce the time your network trains by only training on the hard examples? This paper proposes to select samples with high loss and only train on those in order to speed up training.
Abstract:
This paper introduces Selective-Backprop, a technique that accelerates the training of deep neural networks (DNNs) by prioritizing examples with high loss at each iteration. Selective-Backprop uses the output of a training example's forward pass to decide whether to use that example to compute gradients and update parameters, or to skip immediately to the next example. By reducing the number of computationally-expensive backpropagation steps performed, Selective-Backprop accelerates training. Evaluation on CIFAR10, CIFAR100, and SVHN, across a variety of modern image models, shows that Selective-Backprop converges to target error rates up to 3.5x faster than with standard SGD and between 1.02--1.8x faster than a state-of-the-art importance sampling approach. Further acceleration of 26% can be achieved by using stale forward pass results for selection, thus also skipping forward passes of low priority examples.
Authors: Angela H. Jiang, Daniel L.-K. Wong, Giulio Zhou, David G. Andersen, Jeffrey Dean, Gregory R. Ganger, Gauri Joshi, Michael Kaminksy, Michael Kozuch, Zachary C. Lipton, Padmanabhan Pillai
https://arxiv.org/abs/1910.00762
...
https://www.youtube.com/watch?v=rvr143crpuU
Transaction
Created
1 year ago
Content Type
Language
video/mp4
English
Controlling
VIDEO
OPENA
openai-just-fired-ceo-sam-altman
#openai #chatgpt #samaltman
In a surprise move, OpenAI's board of directors has fired Sam Altman. Chairman Greg Brockman was apparently unaware and has resigned.
Our Discord: https://ykilcher.com/discord
OUTLINE:
0:00 - Intro
1:40 - Sam's statement
2:00 - Greg Brockman resigns
2:55 - Timeline of happenings
4:40 - Microsoft apparently blindsided
6:15 - OpenAI's structure and charter
10:15 - Altman's recent statements on AI advances
11:15 - Disagreements inside OpenAI
14:30 - Backdrop
15:30 - Scoops from Silicon Valley
16:40 - Doomer opinions
18:05 - Let's apply
References:
https://openai.com/blog/openai-announces-leadership-transition
https://twitter.com/sama/status/1725631621511184771
https://twitter.com/growing_daniel/status/1725731742781497732
https://twitter.com/gdb/status/1725667410387378559
https://twitter.com/sama/status/1725742088317534446
https://twitter.com/hellokillian/status/1725799674676936931
https://twitter.com/gdb/status/1725736242137182594
https://blogs.microsoft.com/blog/2023/11/17/a-statement-from-microsoft-chairman-and-ceo-satya-nadella/
https://www.forbes.com/sites/alexkonrad/2023/11/17/openai-investors-blindsided-by-sam-altmans-firing/?sh=2df01c443fc9
https://www.axios.com/2023/11/17/microsoft-openai-sam-altman-ouster
https://openai.com/our-structure
https://www.theinformation.com/articles/before-openai-ousted-altman-employees-disagreed-over-ai-safety
https://www.theinformation.com/articles/three-senior-openai-researchers-resign-as-crisis-deepens
https://twitter.com/RokoMijic/status/1725711519797633037?t=VvxuA-cVMZS1vKOTr0LYHA&s=09
https://twitter.com/thecaptain_nemo/status/1725743009629983215?t=VvxuA-cVMZS1vKOTr0LYHA&s=09
https://twitter.com/lmsysorg/status/1725557251891699733?t=VvxuA-cVMZS1vKOTr0LYHA&s=09
https://twitter.com/abacaj/status/1724850925741768767?t=qYxRZo-wA1cOvrMnbQ9l_A&s=09
https://twitter.com/DrJimFan/status/1724832446393598283?t=qYxRZo-wA1cOvrMnbQ9l_A&s=09
https://twitter.com/karaswisher/status/1725702501435941294
https://twitter.com/karaswisher/status/1725733594310512775
https://twitter.com/ESYudkowsky/status/1725630614723084627
https://openai.com/careers/search
Links:
Homepage: https://ykilcher.com
Merch: https://ykilcher.com/merch
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
LinkedIn: https://www.linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
...
https://www.youtube.com/watch?v=YiS1cJU2zS8
Transaction
Created
5 months ago
Content Type
Language
video/mp4
English