投稿時間:2022-01-22 09:37:00 RSSフィード2022-01-22 09:00 分まとめ(47件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
IT	気になる、記になる…	Tapbots、｢Tweetbot 6.9｣をリリース − 新たなウィジェットやオプション追加	https://taisy0.com/2022/01/22/151037.html	tapbots	2022-01-21 23:54:33
IT	気になる、記になる…	｢Surface Laptop Studio｣、欧州では2月22日に発売	https://taisy0.com/2022/01/22/151033.html	microsoft	2022-01-21 23:35:05
IT	ITmedia 総合記事一覧	[ITmedia ビジネスオンライン] 1回のキャンプでかける予算はいくら？　ソロキャンプは「1万円以内」が最多	https://www.itmedia.co.jp/business/articles/2201/22/news032.html	itmedia	2022-01-22 08:30:00
AWS	lambdaタグが付けられた新着投稿 - Qiita	Lambda(SAM)からAmazon SNSトピックへpublishを行う方法メモ	https://qiita.com/KWS_0901/items/2c21f11c789d8da91b97	LambdaSAMからAmazonSNSトピックへpublishを行う方法メモLocalStack上で動くAmazonSNSのトピックに対して、AWSSAMCLIで動くLambdaからpublishを行う方法をメモする。	2022-01-22 08:30:19
python	Pythonタグが付けられた新着投稿 - Qiita	【Project Euler】Problem 44: 五角数	https://qiita.com/masa0599/items/71d76aaebd1509dbf3b9	【ProjectEuler】Problem五角数本記事はProjectEulerの「番以下の問題の説明は記載可能」という規定に基づいて回答のヒントが書かれていますので、自分である程度考えてみてから読まれることをお勧めします。	2022-01-22 08:44:29
python	Pythonタグが付けられた新着投稿 - Qiita	【Project Euler】Problem 43: 素数の倍数の部分列	https://qiita.com/masa0599/items/e6ff607cca4300cfc097	【ProjectEuler】Problem素数の倍数の部分列本記事はProjectEulerの「番以下の問題の説明は記載可能」という規定に基づいて回答のヒントが書かれていますので、自分である程度考えてみてから読まれることをお勧めします。	2022-01-22 08:42:47
python	Pythonタグが付けられた新着投稿 - Qiita	【Project Euler】Problem 42: 符号化された三角数	https://qiita.com/masa0599/items/4f96c1987ccc95a58b33	【ProjectEuler】Problem符号化された三角数本記事はProjectEulerの「番以下の問題の説明は記載可能」という規定に基づいて回答のヒントが書かれていますので、自分である程度考えてみてから読まれることをお勧めします。	2022-01-22 08:40:49
AWS	AWSタグが付けられた新着投稿 - Qiita	Lambda(SAM)からAmazon SNSトピックへpublishを行う方法メモ	https://qiita.com/KWS_0901/items/2c21f11c789d8da91b97	LambdaSAMからAmazonSNSトピックへpublishを行う方法メモLocalStack上で動くAmazonSNSのトピックに対して、AWSSAMCLIで動くLambdaからpublishを行う方法をメモする。	2022-01-22 08:30:19
AWS	AWSタグが付けられた新着投稿 - Qiita	LocalStack を用いたAmazon SNS動作確認方法メモ	https://qiita.com/KWS_0901/items/aa9878cb8266514bf15e	LocalStackを用いたAmazonSNS動作確認方法メモLocalStack上で動作するAmazonSNSをAWSCLIで操作し、動作確認する。	2022-01-22 08:24:10
AWS	AWSタグが付けられた新着投稿 - Qiita	Lambda(SAM)からAmazon CloudWatch Logs(LocalStack) のログイベントを取得する方法 メモ	https://qiita.com/KWS_0901/items/c19fb4398d38460de900	awslogscreateloggrouploggroupnametestloggroupendpointurlhttplocalhostprofilelocalstackログストリームを作成するロググループtestloggroupにログストリームtestlogstreamを作成する。	2022-01-22 08:21:46
AWS	AWSタグが付けられた新着投稿 - Qiita	LocalStack を用いたAmazon CloudWatch Logs ロググループ作成方法 メモ	https://qiita.com/KWS_0901/items/e4a20a5366088f585340	ロググループ作成方法AWSCLIを使用し、ロググループを作成する。	2022-01-22 08:18:27
Docker	dockerタグが付けられた新着投稿 - Qiita	LocalStack を用いたAmazon SNS動作確認方法メモ	https://qiita.com/KWS_0901/items/aa9878cb8266514bf15e	LocalStackを用いたAmazonSNS動作確認方法メモLocalStack上で動作するAmazonSNSをAWSCLIで操作し、動作確認する。	2022-01-22 08:24:10
Docker	dockerタグが付けられた新着投稿 - Qiita	LocalStack を用いたAmazon CloudWatch Logs ロググループ作成方法 メモ	https://qiita.com/KWS_0901/items/e4a20a5366088f585340	ロググループ作成方法AWSCLIを使用し、ロググループを作成する。	2022-01-22 08:18:27
海外TECH	Ars Technica	Picard and Guinan have a warm reunion in S2 trailer for Star Trek: Picard	https://arstechnica.com/?p=1827532	picard	2022-01-21 23:14:47
海外TECH	Ars Technica	Unvaccinated 5X more likely to get omicron than those boosted, CDC reports	https://arstechnica.com/?p=1827676	booster	2022-01-21 23:14:41
海外TECH	DEV Community	Sending Custom emails with NodeJS from 🪄scratch and for no cost whatsoever🗡	https://dev.to/ironcladdev/sending-custom-emails-with-nodejs-from-scratch-and-for-no-cost-whatsoever-1i2n	Sending Custom emails with NodeJS from 🪄scratch and for no cost whatsoeverIn this tutorial I m going to show you how you can send custom emails with just NodeJS two sweet npm packages and a single email address The best part No email hosting is required you can do this for no cost at all and all in under lines of code Let s get started In this example I ll be using gmail to get an email You may do this anywhere else but I would prefer google due to how easy it is to get an alternate email account After signing up for google go to go to the Security section and scroll to the Signing in to Google section To be able to send emails you ll need an app password from your google account You can obtain this after setting up two step verification If you are not using google use the password for that email account and keep it somewhere safe since we ll need it again After turning on two step verification click on App Passwords and create a new one for Mail As for the device click on Other custom name and name it anything you want After that copy the digit app password save it and we re ready to start getting into the code First install two packages from npm nodemailer and nodemailer juiceconst nodemailer require nodemailer const inLineCss require nodemailer juice Why do we need nodemailer juice Emails typically only accept inline styles and not any styling from CSS If you don t want to sweat your brain to pieces it s a good option to use nodemailer juice it automatically converts css within lt style gt lt style gt tags into inline CSS for you Now for the emailing function I ve already made it so all you have to do is call it to send an email function sendEmail to subject message let transporter nodemailer createTransport service gmail change if not using gmail host smtp gmail com also change if not using gmail port secure true auth user lt you email com gt pass lt your app password gt transporter use compile inLineCss makes your email nice and full of css let mailDetails from lt you email com gt to to subject subject html message transporter sendMail mailDetails function err data if err console error err For the sendEmail function you will have to pass in an html document for the parameter message Just to let you know hover states css listeners and literally anything that won t run in inline styles will not run Unfortunately I m too lazy to write an entire document here for you I do have a template email at IroncladDev Noseletter in case you wanted to use it To make things a bit easier I would store the html document in a function as a template string and have some parameters in which I could pass title body and some other options const emailTemplate title body gt lt DOCTYPE html gt lt html lang en gt lt head gt lt style gt lt style gt lt head gt lt body gt lt h gt title lt h gt lt p gt body lt p gt lt body gt lt html gt To pass the email template function into the sendEmail function simply do like so sendEmail someone email com Email Title emailTemplate Email Title This is the body of the email That my friend is all it takes to send emails in NodeJS Liked this post Don t forget to subscribe to my newsletter located on my website footer for new posts new projects recaps and more Join my discord to get in touch and hang out Also be sure to support this post with a couple reactions	2022-01-21 23:34:43
海外TECH	Engadget	'We Met in Virtual Reality’ finds love in the metaverse	https://www.engadget.com/we-met-in-virtual-reality-review-sundance-vrchat-234048235.html?src=rss	x We Met in Virtual Reality finds love in the metaverseMark Zuckerberg s vision of a sanitized hypercapitalist metaverse will likely never be as compelling or idiosyncratic as VRChat the virtual reality community that s been home to anime fans Furries and a slew of other sub cultures since That s my main takeaway from We Met in Virtual Reality the first documentary filmed entirely in VRChat which premiered at the Sundance Film Festival today There s no chance Zuck s metaverse would let people wear trademarked avatars without paying a ton attend exotic clubs to receive or give virtual lapdances or allow users to build whatever the hell they want VRChat as portrayed by director Joe Hunting is basically a proto metaverse where anything is possible And for many it has served as a crucial social hub during the pandemic a place where they can forget about the world relax with friends and maybe find love But of course that s been the nature of practically every online community We re social animals ーpeople have always been able to connect with each other over BBS IRC Usenet and the plethora of forums and chat services that populated the early internet I spent most of the s hanging out in anime and gaming chat rooms the sorts of places that today s connected youth would probably find quaint Still the people I met there helped me survive the worst parts of middle and high school Those relationships and the internet itself shaped me into who I am for better or worse We Met in Virtual Reality proves that the unbridled experimental sense of online community is still alive and well today despite relentless consolidation from Big Tech But now instead of staring at tiny CRT monitors people are slapping on VR headsets to explore fully realized environments Hardcore VRChat users are also investing in powerful computing rigs as well as upgrades like finger and whole body tracking In the s I was grateful to get another MB of RAM so that I could have more than one browser window open Today VRChat devotees can communicate using American Sign Language or have their anime avatars show off their belly dancing skills Hunting approaches his subjects with the eye of an anthropologist without any judgment towards their sometimes ridiculous avatars do all the anime ladies need to have jiggly Dead or Alive level boob physics We Met in Virtual Reality begins as a chill hangout flick ーwe follow a group of friends as they have virtual drinks and go on joyrides in crudely built VR cars ーbut it quickly moves beyond the novelty of its setting One person credits their VRChat girlfriend for helping them to quot unmute quot after being silent for two years An exotic performer explains that being able to dance for people in VRChat helped her grieve with a family tragedy and manage a bout of alcoholism Joe HuntingThe film chronicles how that exotic dancer a young woman based in the UK formed a romantic relationship with another VRChat user in Miami These sorts of cyber relationships aren t anything new but the VR platform allowed them to do much more than trade links and memes over IM They could exist in a space together go on dates to new environments every night I won t spoil where things end up for the couple but I can say that it wouldn t have been nearly as effective outside of VR We Met in Virtual Reality effectively conveys why people would gravitate towards VRChat especially during a pandemic But it doesn t fully capture the wonder of exploring these environments yourself Seeing people hop on a virtual rollercoaster isn t nearly as thrilling as doing it where your entire field of vision is covered and you can easily get vertigo But I don t blame Hunting too much for that his job was to boil down the VR experience so people can enjoy it on a D screen and the film is mostly successful in that respect The film was shot using a virtual camera that could mimic all of the functionality of a typical shooter from focus points to aperture levels So even though it s produced in an alien environment most people aren t familiar with it still feels like a traditional documentary Hunting has spent the past few years making VR documentaries starting with a few short films as well as the series Virtually Speaking It s clear from We Met in Virtual Reality that he s not just dropping into the community for a quick story Instead he sees the humanity behind the avatars and virtual connections These people aren t just escaping from their lives nbsp with VR ーtheir lives are being made richer because of it	2022-01-21 23:40:48
海外科学	NYT > Science	As Omicron Crests, Booster Shots Are Keeping Americans Out of Hospitals	https://www.nytimes.com/2022/01/21/health/covid-boosters-cdc-omicron.html	As Omicron Crests Booster Shots Are Keeping Americans Out of HospitalsExtra vaccine doses were expected to lower infection rates But the shots also seem to be preventing severe illness caused by the new variant the C D C reported	2022-01-21 23:37:39
金融	金融総合：経済レポート一覧	主要銀行貸出動向アンケート調査（2022年1月）	http://www3.keizaireport.com/report.php/RID/482006/?rss	主要銀行貸出動向アンケート調査	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	金融政策決定会合議事要旨（2021年12月16、17日開催分）	http://www3.keizaireport.com/report.php/RID/482007/?rss	日本銀行	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	暗号資産「ゲーム仮想通貨」の衝撃～あなたの知らないゲームの世界でやりとりされる暗号資産の話：Watching	http://www3.keizaireport.com/report.php/RID/482014/?rss	watching	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	インドの不良債権比率は中銀予想に反し低下 ～背景に機動的な政策と想定を超えるペースの景気回復：Mizuho RT Express	http://www3.keizaireport.com/report.php/RID/482015/?rss	mizuhortexpress	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	FX Daily（1月20日）～ドル円、一時114円割れ	http://www3.keizaireport.com/report.php/RID/482016/?rss	fxdaily	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	ECBの12月政策理事会のAccount～flexibility：井上哲也のReview on Central Banking	http://www3.keizaireport.com/report.php/RID/482017/?rss	accountflexibility	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	SECによる株取引アプリの規制に関わる議論～証券会社によるAIや予測分析に関する問題へのコメント募集を実施：証券・金融取引の法制度	http://www3.keizaireport.com/report.php/RID/482019/?rss	大和総研	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	インドネシア中銀、米FRBの動きを警戒して預金準備率引き上げへ～ペリー総裁は米FRBが3月利上げ開始と予想、当面はルピア相場と金利安定を重視する展開へ：Asia Trends	http://www3.keizaireport.com/report.php/RID/482021/?rss	asiatrends	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	令和４年度の年金額改定について～年金額は昨年度から0.4％の引き下げ	http://www3.keizaireport.com/report.php/RID/482027/?rss	厚生労働省	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	トルコリラの落ち着きは本物か、それとも「嵐の前の静けさ」か～当局の取り組みは有効性に疑問、状況が一変する可能性には引き続き注意が必要な展開：Asia Trends	http://www3.keizaireport.com/report.php/RID/482034/?rss	asiatrends	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	米利上げ開始秒読みでも円高ドル安反応のワケ～マーケット・カルテ２月号	http://www3.keizaireport.com/report.php/RID/482036/?rss	円高ドル安	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	データを読む：相次ぐ航空機リースを巡るトラブル、投資する中小企業にも影響か	http://www3.keizaireport.com/report.php/RID/482043/?rss	中小企業	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	2020年度株式分布状況調査の調査結果について＜レポート編＞	http://www3.keizaireport.com/report.php/RID/482049/?rss	日本取引所グループ	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	金融政策のリアル～「共通テスト」で読み解く異次元緩和	http://www3.keizaireport.com/report.php/RID/482058/?rss	異次元緩和	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	交換できるくん（東証マザーズ）～住宅設備機器と交換工事をセットで販売するeコマース事業を展開。EC化率の上昇等を想定し、中長期的な業績拡大を予想：アナリストレポート	http://www3.keizaireport.com/report.php/RID/482060/?rss	東証マザーズ	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	マンスリー通貨レポート（2022年1月）～為替レートと為替市場における当面のテーマ	http://www3.keizaireport.com/report.php/RID/482061/?rss	為替レート	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	FX Weekly（2022年1月21日号）～来週の為替相場見通し（1）ドル円：更なるタカ派化を懸念するリスク回避ムード	http://www3.keizaireport.com/report.php/RID/482063/?rss	fxweekly	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	投資INSIDE-OUT vol.180「コロナ禍における消費者意識の変化～マクロ経済データを読み解く(13)～」	http://www3.keizaireport.com/report.php/RID/482066/?rss	insideoutvol	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	【石黒英之のMarket Navi】政策転換機運が強まりつつある中国～中国企業を見直す機運が高まる...	http://www3.keizaireport.com/report.php/RID/482067/?rss	marketnavi	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	らくっと通信 第4回 インフレについて考えてみよう	http://www3.keizaireport.com/report.php/RID/482068/?rss	野村アセットマネジメント	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	【注目検索キーワード】アニマルウェルフェア	http://search.keizaireport.com/search.php/-/keyword=アニマルウェルフェア/?rss	検索キーワード	2022-01-22 00:00:00
金融	金融総合：経済レポート一覧	【お薦め書籍】5秒でチェック、すぐに使える! 2行でわかるサクサク仕事ノート	https://www.amazon.co.jp/exec/obidos/ASIN/4046053631/keizaireport-22/	結集	2022-01-22 00:00:00
海外ニュース	Japan Times latest articles	Magnitude 6.6 earthquake hits western Japan	https://www.japantimes.co.jp/news/2022/01/22/national/miyazaki-oita-earthquake/	massive	2022-01-22 08:12:15
ニュース	BBC News - Home	The Paper: PM beefs up team and NHS staff jab 'U-turn talks'	https://www.bbc.co.uk/news/blogs-the-papers-60092388?at_medium=RSS&at_campaign=KARANGA	parties	2022-01-21 23:24:22
LifeHuck	ライフハッカー［日本版］	賞味期限や締切を管理できる！ キングジムの長期タイマー「リミッツ」	https://www.lifehacker.jp/article/roomie-kingjim-limits/	賞味期限	2022-01-21 23:30:00
北海道	北海道新聞	国連、保有国に核軍縮の努力要求　グテレス氏、核禁条約発効１年	https://www.hokkaido-np.co.jp/article/636494/	事務総長	2022-01-22 08:09:00
北海道	北海道新聞	アイルランドも規制解除　新変異株「嵐切り抜けた」	https://www.hokkaido-np.co.jp/article/636493/	新型コロナウイルス	2022-01-22 08:09:00
仮想通貨	BITPRESS（ビットプレス）	[日経] ニューヨーク市長、初の給与は仮想通貨で　宣言実行	https://bitpress.jp/count2/3_9_13007	給与	2022-01-22 08:25:12
GCP	Cloud Blog	PyTorch/XLA: Performance debugging on Cloud TPU VM: Part III	https://cloud.google.com/blog/topics/developers-practitioners/pytorchxla-performance-debugging-cloud-tpu-vm-part-iii/	PyTorch XLA Performance debugging on Cloud TPU VM Part IIIThis article is the final in the three part series to explore the performance debugging ecosystem of PyTorch XLA on Google Cloud TPU VM In the first part we introduced the key concept to reason about the training performance using PyTorch XLA profiler and ended with an interesting performance bottleneck we encountered in the Multi Head Attention MHA implementation in PyTorch In the second part we began with another implementation of MHA introduced in PyTorch which solved the performance bottleneck and identified an example of Dynamic Graph during evaluation using the profiler and studied a possible trade off between CompileTime penalty with device to host transfers In this part we shift gears to server side profiling Recall from client and server terminology introduced in the first part the server side functionality refers to all the computations that happen at TPU Runtime aka XRT server and beyond on TPU device Often the objective of such an analysis is to peek into performance of a certain op or a certain section of your model on TPUs PyTorch XLA facilitates this with user annotations which can be inserted in the source code and can be visualized as a trace using Tensorboard Environment setupWe continue to use the mmf multimodal framework example used in the first two parts If you are starting with this part please refer to the Environment Setup section of part to create a TPU VM and continue from the TensorBoard Setup described in this post We recap the commands here for easy accessCreation of TPU VM instance if not already created Here is the command to SSH into the instance once the instance is created TensorBoard setupNotice the ssh flag argument for ssh tunneled port forwarding This allows you to connect to the application server listening to port via localhost on your own browser To setup TensorBoard version correctly and avoid conflict with other local version please uninstall the existing TensorBoard version first In the current context the application server is the TensorBoard instance listening to port This will start the TensorBoard server accessible via localhost on your browser Notice that no profiling data is collected yet Follow the instructions from the user guide to open the PROFILE view localhost Training setupWe anchor to the PyTorch XLA environment for this case study Update alternative make python default Configure environment variables MMF training environmentThe MMF Multimodal Training Framework library developed by Meta Research is built to help researchers easily experiment with the models for multi modal text image audio learning problems  As described in the roadmap we will use the UniTransformer model for this case study We will begin by cloning and installing the mmf library specific hash chosen for reproducibility purposes Before we install the mmf library in developer mode please make the following modifications in the requirement txt such that the existing PyTorch environment is not overridden when mmf is installed Apply the following patch for validate batch sizes method Specific to the commit selected for this article Install the mmf library in developer mode Trace APIThe PyTorch XLA profiler provides two primary APIs StepTrace Trace  to help you analyze the different segments of your code through annotations which can later be visualized using TensorBoard To use StepTrace or Trace first a profiler server needs to be started Starting profiler serverIn the following code snippet we modified the main method mmf cli run py to introduce the start server lt port number gt call Notice that the server object returned by this method will only persist if captured in a variable Therefore server xp start server will start a profiler server that persists throughout the training However if we only called start server xp start server without any assignment the server object does not persist and you will not be able to interact with the server later on we will request the server via TensorBoard to capture the profile for a given period of time Make a note of the port number used as the argument to start server call in this case We will use this when communicating with the profiler server in the later sections Insert traces annotationsOnce the mmf library is installed in the developer mode we can make changes to the source code as shown below to follow this case study In the following code snippet we illustrate StepTrace and Trace annotations The former introduces a named context called Training Step within which we introduce a few more nested named contexts to capture forward backward and update times through the latter annotations Note that xp refers to import torch xla debug profiler as xp as illustrated in earlier code snippet Once the training starts we can capture these annotations onto the trace by either the xp trace method or TensorBoard Profiler s Capture Profile feature In this case study we will use the TensorBoard feature for interactive and iterative profile capture In development test settings xp trace can be used to capture the profile data into a TensorBoard logdir however be cautioned that trace data can quickly grow and therefore trace call should only be active for a reasonable time period Start TensorBoardRecall the TensorBoard setup instructions shared in the earlier section  TensorBoard Setup You started the TensorBoard server at port And when you ssh into the TPU VM instance we forwarded port of our localhost to of the TPU VM Once you open localhost in your browser you will see the following UI Note If you do not see Profile options in the drop down menu please check the TensorBoard version and ensure that the profiler plugin was installed as instructed in the setup section Start trainingStart the training using the same script that you used earlier with one modification We change the distributed world size to for all reduce visualization in the trace Once the training is underway you should begin to see the following log snippet Capture profileForm TensorBoard UI which you started earlier Click Capture Profile and specify the localhost the same port number you specified in the start server call Once the capture is complete you should see an overview page similar to the following if more than one training step is captured in the trace else the overview page may be blank since it requires at least one training step completed to provide any input pipeline recommendations In this case study we will focus on the trace view for details about other views For memory profiler pod viewer etc please refer to the TensorBoard profiler documentation Some of these functionalities are not fully supported by TPU VM at the time of writing this article Navigate to the annotations in the Trace ViewerIn the Trace Viewer Traces collected from all the device i e TPU processes and all the host i e CPU processes are displayed You can use the search bar and scroll down to find the annotations you have introduced in the source code Understanding TraceThe following picture shows the annotations that we created in the code snippet example Notice the Training Step trace and sub traces forward and backward pass These annotations on the CPU process trace Indicate the time spent in the forward backward pass IR graph traversal In case forward or backward passes force early execution unlowered op or value fetch you will notice forward or backward traces broken and interspersed multiple times with StepMarker StepMarker is an in built annotation which corresponds to implicit or explicit mark step call Notice also the FINISH UPDATE trace In the mmf code this corresponds to reduce gradients and update operations We notice another host process which which is starting TPUExecute function call If we create a window from the TPUExecute to the end of the RunExecutable trace we expect to see graph execution and all reduce traces on the device scroll back up to the top Notice the all reduce xla op and corresponding all reduce cross replica sum appearing in tensorflow op Tensorflow Name Scope and Tensorflow Ops show up in the trace when XLA HLO DEBUG is enabled ref code snippet in Start Training section It shows the annotation propagated to the HLO High Level Operations graph Notice also the gaps on the device trace The duty cycle i e the fraction of annotated trace per cycle is the fraction of time spent on graph execution on the device To investigate the gaps it s often a good practice to select the gap duration using the trace viewer window tool and then examine the CPU traces to understand what other operations are being executed in that period For example in the following trace we select the ms period where no device trace is observed Examining the same period across cpu traces zoomed in view we notice three main time chunks Overlapping with the IR generation backward pass of the next step followed by StepMarker mark step call MarkStep signals to PyTorch XLA that the IR graph traversed thus far is to be executed now If it s a new graph it will be compiled and optimized first you will notice longer StepMarker trace and also separate process traces annotated “xxyy Graph Data is Transferred to the Server i e transfer from host to TPU device In the case of TPU VM transfer to server is expected to be an inexpensive operation XRTExecute is initiated which loads the program graph from cache if found or loads the input graph and eventually triggers TPU execution Details of XRTExecute allocation load program signature etc is outside the scope of this case study Notice also that the IR generation for the following step has already begun while the current step is executed on TPU In summary the gaps in device trace are good regions to investigate Apart from the three chunks or scenarios above it can also be due to in efficient data pipeline In order to investigate you can also add annotations to data loading parts of the code and then inspect it in the trace wrt to device execution For a good pipeline data loading and transfer to server should overlap with device execution If this is not the case the device will be idle for some time waiting for data and this would be input pipeline bottleneck Annotation propagationSince PyTorch XLA training involves IR graph translated into HLO graph which is then compiled and optimized further by the runtime and those optimization do not preserve the initial graph structure these annotations when propagated to HLO graphs and beyond enabled by XLA HLO DEBUG do not appear in the same order For example consider the following trace zooming into the trace shown above This window does not correspond to multiple forward and backward traces in the host process you notice gaps in the trace when neither forward or train step tags appear still we notice Training Step Forward and FINISH UPDATE traces are interspersed here And this is because the graph to which initial annotation is propagated has undergone multiple optimization passes ConclusionAnnotation propagation into the host and device traces using the profiler API provides a powerful mechanism to analyze the performance of your training In this case study we did not deduce any optimizations to reduce training time from this analysis but only introduced it for educational purposes These optimizations can improve efficiency TPU utilization duty cycle training ingestion throughput and such Next stepsAs an exercise we recommend the readers to repeat the analysis performed in this study with log interval set to This exercise will accentuate device to host transfer costs and the effects of some of the changes discussed to optimize the training performance  This concludes the part of our three part series on performance debugging We reviewed the basic performance concepts and command line debug in part In Part we investigated a transformer based model training which was slow due to too frequent device to host transfers Part ended with an exercise for the reader s to improve the training performance In Part we started with the solution of the part exercise and examined another common pattern of performance degradation the dynamic graph And finally in this post we took a deeper dive into performance analysis using annotations and tensorboard trace viewer We hope that the concepts presented in this series would be helpful for the readers to debug their training runs effectively using PyTorch XLA profiler and derive actionable insights to improve the training performance The reader is encouraged to experiment with their own models and apply and learn the concepts presented here  AcknowledgementsThe author would like to thank Jordan Totten Rajesh Thallam Karl Weinmeister and Jack Cao Google for your patience and generosity Each one of you provided numerous feedback without which this post would be riddled with annoying errors Special thanks to Jordan for patiently testing the case studies presented in this series for finding many bugs and providing valuable suggestions Thanks to Joe Spisak Geeta Chauhan Meta and Shauheen Zahirazami and Zak Stone Google for your encouragement especially for the feedback to split what would have been too long a post for anyone to read Thanks to Ronghong Hu Meta AI for the feedback and questions on the early revisions which made especially the trace discussion much more accessible and helpful And finally to Amanpreet Singh Meta AI the author of MMF framework for your inputs in various debugging discussions case study selection and for your inputs in building the TPU support in MMF without your help this series would not be possible Related ArticlePyTorch XLA Performance debugging on Cloud TPU VM Part IIIn this blog post we build upon the concepts introduced in part and apply these to analyze and improve performance for a case study in Read Article	2022-01-21 23:30:00