投稿時間:2021-09-11 09:43:51 RSSフィード2021-09-11 09:00 分まとめ(60件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
IT	ITmedia 総合記事一覧	[ITmedia News] Epic対Apple訴訟に判決　Appleにアプリ内購入の強制禁止命令もEpicは控訴か	https://www.itmedia.co.jp/news/articles/2109/11/news031.html	apple	2021-09-11 08:06:00
AWS	AWS	Enable Amazon DevOps Guru for AWS CloudFormation Stacks | Amazon Web Services	https://www.youtube.com/watch?v=lwiLev0Vn-c	Enable Amazon DevOps Guru for AWS CloudFormation Stacks Amazon Web ServicesIn this video you ll see how to enable Amazon DevOps Guru for CloudFormation stacks With this capability you can review operational performance data from a single dashboard and leverage machine learning capabilities to identify anomalous application behavior For more information on this topic please visit the resource below Amazon DevOps Guru Easily configure Amazon DevOps Guru across multiple accounts and Regions using AWS CloudFormation StackSets Subscribe More AWS videos More AWS events videos ABOUT AWSAmazon Web Services AWS is the world s most comprehensive and broadly adopted cloud platform offering over fully featured services from data centers globally Millions of customers ーincluding the fastest growing startups largest enterprises and leading government agencies ーare using AWS to lower costs become more agile and innovate faster AWS AmazonWebServices CloudComputing	2021-09-10 23:22:15
AWS	AWS	Create Custom Ticket Systems for Amazon DevOps Guru Notifications | Amazon Web Services	https://www.youtube.com/watch?v=Mu8IqWVGUfg	Create Custom Ticket Systems for Amazon DevOps Guru Notifications Amazon Web ServicesIn this video you ll see how you can create custom ticket systems for Amazon DevOps Guru notifications With this capability you can natively integrate DevOps Guru with AWS Systems Manager to create OpsItems for the insights that are generated or integrate DevOps Guru with third party incident management tools from PagerDuty and Atlassian OpsGenie to automatically manage incidents within those platforms To learn more about Amazon DevOps Guru visit Subscribe More AWS videos More AWS events videos ABOUT AWSAmazon Web Services AWS is the world s most comprehensive and broadly adopted cloud platform offering over fully featured services from data centers globally Millions of customers ーincluding the fastest growing startups largest enterprises and leading government agencies ーare using AWS to lower costs become more agile and innovate faster AWS AmazonWebServices CloudComputing	2021-09-10 23:11:21
AWS	AWS	ConnectCareHero Leverages Amazon CodeGuru to Automate Code Reviews | Amazon Web Services	https://www.youtube.com/watch?v=Zk7hWiwWf0g	ConnectCareHero Leverages Amazon CodeGuru to Automate Code Reviews Amazon Web ServicesConnectCareHero s digital platform provides access to live culturally relevant activity programming to assist senior living organizations in tackling social isolation loneliness at scale Using Amazon CodeGuru has allowed ConnectCareHero to accelerate code reviews for thousands of lines of code fix security vulnerabilities and meet coding best practices Learn more about Amazon CodeGuru at Subscribe More AWS videos More AWS events videos ABOUT AWSAmazon Web Services AWS is the world s most comprehensive and broadly adopted cloud platform offering over fully featured services from data centers globally Millions of customers ーincluding the fastest growing startups largest enterprises and leading government agencies ーare using AWS to lower costs become more agile and innovate faster AWS AmazonWebServices CloudComputing	2021-09-10 23:01:29
python	Pythonタグが付けられた新着投稿 - Qiita	【Python】SBI証券に自動ログインしてみた	https://qiita.com/hirockio2206/items/540a0991ba29dec3fc40	証券会社のHPへログインした後に毎日確認したことはありますかそんな第一歩としてまずは自動ログインしてみました。	2021-09-11 08:04:39
Program	[全てのタグ]の新着質問一覧｜teratail（テラテイル）	FlutterのAPKのbuildでKeytoolExceptionが発生する。(Android)	https://teratail.com/questions/358884?rss=all	FlutterのAPKのbuildでKeytoolExceptionが発生する。	2021-09-11 08:39:10
Program	[全てのタグ]の新着質問一覧｜teratail（テラテイル）	phpでmysqlを使用し「登録」ボタンを押してデータの反映をさせたい	https://teratail.com/questions/358883?rss=all	phpでmysqlを使用し「登録」ボタンを押してデータの反映をさせたいphpで会員登録ボタンを押すとPOSTでデータが送られmysqlにデータが入る様にしたいのですが、何度実行してもmysqlのテーブルテーブル名tableに入力データが反映されません。	2021-09-11 08:06:05
Program	[全てのタグ]の新着質問一覧｜teratail（テラテイル）	Rails tutorial のrails test でエラー発生	https://teratail.com/questions/358882?rss=all	Railstutorialのrailstestでエラー発生前提・実現したいことMチップ搭載のmacbookにてRailsnbsptutorialを学習しております。	2021-09-11 08:05:15
Ruby	Rubyタグが付けられた新着投稿 - Qiita	【mariaDB】データベースの中身の確認について	https://qiita.com/kawadesu88/items/c406b3b372b7c271bd5c	ローカルでのデータはすぐ目視で確認できていたので、いざ本番環境のデータを見るとなったときに「どこ見に行けばいいの」と思ったので作成しました。	2021-09-11 08:43:51
AWS	AWSタグが付けられた新着投稿 - Qiita	【AWS】AWSを勉強していく中で「？」ってなった単語をまとめていく② -EC2編-	https://qiita.com/nao0725/items/80d52a49f158c9e4cdc1	【AWS】AWSを勉強していく中で「」ってなった単語をまとめていく②EC編前回の続き目的自分用のメモ単語の意味と役割を掴んでイメージを把握するECを運用する上で出てきた単語ECAmazonElasticComputeCloudAWSのクラウド上でサーバーを実行するためのサービス単一ではなく複数の仮想サーバーインスタンスを実行するECを利用することで、OSを乗せた仮想環境をクラウド上にすばやく作ることができるインスタンスタイプ用途別に集められた「インスタンスファミリー」と、CPUなどのスペックを示す「インスタンスサイズ」から構成されているインスタンスファミリーの数字は「世代」を表していて、数字が大きければ大きいほど新しいインスタンスファミリー「汎用」「コンピューティング最適化」「メモリ最適化」「高速コンピューティング」「ストレージ最適化」の種類に分類されるそれぞれのシステムによって設定するセキュリティグループECインスタンスなどに適用するファイアウォール機能で、主にVPCリソースのトラフィックを制御するのに使われるECインスタンスへのアクセスを許可したり、トラフィックを制御する役割があるファイアーウォールとはネットワークの門番のことウイルス等から守るインバウンドルールセキュリティグループに関連付けられたインスタンスにアクセスできるルールアウトバウンドルールセキュリティグループに関連付けられたインスタンスからどの送信先にトラフィックを送信できるかトラフィックの送信先と送信先ポートを制御するルールネットワークACLサブネットに属するすべてのインスタンスに対して適用されるファイアウォール許可と拒否のルールを定義するECのIPアドレスの種類パブリックIPアドレスインターネットから接続できるアドレスインスタンスを再開しても同じIPアドレスは付与されないプライベートIPアドレスECインスタンスにアタッチされているすべてのENIに対して、個別に付与できるIPアドレスインスタンスを再開したときに同じIPアドレスのままになるENIElasticNetworkInterface仮想ネットワークインターフェースのことインスタンスにはデフォルトでつのENIがアタッチ仮想化環境に追加することされており、必要に応じて複数のENIをアタッチできます。	2021-09-11 08:02:57
Azure	Azureタグが付けられた新着投稿 - Qiita	Azureで基本的な構成を構築してみる③（MySQL作成, VMからのSSL接続）	https://qiita.com/MAKOTO1995/items/bab048556921f10b4dac	Azureで基本的な構成を構築してみる①前提、ネットワーク・VM作成Azureで基本的な構成を構築してみる②ALB、DNS作成Azureで基本的な構成を構築してみる③MySQL作成VMからのSSL接続←今ここアーキテクチャ図今回はDBとしてMySQLを作成し、可能な限りVMからセキュアな接続設定をしていきます。	2021-09-11 08:33:32
Azure	Azureタグが付けられた新着投稿 - Qiita	Azureで基本的な構成を構築してみる②（ALB、DNS作成）	https://qiita.com/MAKOTO1995/items/8276122169d727d81d96	DNSゾーン作成Azureポータルで「DNSゾーン」に移動→「作成」【基本】タブ・リソースグループVMAGW等と同じリソースグループ選択・名前使用する独自ドメインを入力ここがゾーン名となる→作成Aレコード登録作成したゾーンに移動し、Aレコードエイリアスレコードとして、ターゲットに上記で作成したAGWのパブリックIPを選択して保存します。	2021-09-11 08:32:46
Azure	Azureタグが付けられた新着投稿 - Qiita	Azureで基本的な構成を構築してみる①（前提、ネットワーク・VM作成）	https://qiita.com/MAKOTO1995/items/a838e70918eefca98dfa	・仮想ネットワーク、サブネット前述で作成済のものを選択・パブリックIP新規作成し、「静的」を選択静的IPは追加で月程度課金される・パブリック受信ポートHTTPSSHが選択されていることを確認【管理、詳細、タグ】デフォルトのまま設定→作成後、SSH秘密鍵を忘れずダウンロードしておくVM内設定作成したVMにSSHログインしてネットワーク設定を行なっていきます。	2021-09-11 08:30:51
Ruby	Railsタグが付けられた新着投稿 - Qiita	【mariaDB】データベースの中身の確認について	https://qiita.com/kawadesu88/items/c406b3b372b7c271bd5c	ローカルでのデータはすぐ目視で確認できていたので、いざ本番環境のデータを見るとなったときに「どこ見に行けばいいの」と思ったので作成しました。	2021-09-11 08:43:51
Ruby	Railsタグが付けられた新着投稿 - Qiita	[メモ] Docker「ruby:3.0-slim-buster」に Rails6 (簡素版) を導入してみる (Django使用経験あり)	https://qiita.com/robozushi10/items/79b3d39583b47da76449		2021-09-11 08:37:29
Apple	AppleInsider - Frontpage News	Prototype reveals Apple considered cellular Apple Watch Series 2	https://appleinsider.com/articles/21/09/10/prototype-reveals-apple-considered-cellular-apple-watch-series-2?utm_medium=rss	Prototype reveals Apple considered cellular Apple Watch Series A supposed Apple Watch Series prototype reveals Apple was working to introduce a cellular version of its wearable in and experimenting with gold stainless steel casings Apple debuted cellular capable variants of Apple Watch in with the launch of Apple Watch Series but earlier rumblings claimed the company planned to introduce the technology with Apple Watch Series It appears that Apple was indeed working toward integration of LTE in a second generation device as evidenced by photos and information of a prototype unit shared on Twitter by DongleBookPro The user is known for their collection of prototype and unreleased Apple devices and has in the past shed light on previously unknown or rumored internal projects Read more	2021-09-10 23:27:45
海外科学	NYT > Science	Why Federal Research Bolsters the Case for Coronavirus Vaccine Mandates	https://www.nytimes.com/2021/09/10/health/coronavirus-vaccine-mandate.html	Why Federal Research Bolsters the Case for Coronavirus Vaccine MandatesScientists believe the administration s new measures may tamp down the pandemic although the effects will not immediately be obvious	2021-09-10 23:44:26
金融	金融総合：経済レポート一覧	デリバティブ取引に関する定例市場報告（2021年6月末）	http://www3.keizaireport.com/report.php/RID/467973/?rss	日本銀行	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	FX Daily(9月9日)～ドル円、109円台後半まで下落	http://www3.keizaireport.com/report.php/RID/467974/?rss	fxdaily	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	テーマはインフレのバトン ～中古車→家賃→労働コスト：Market Flash	http://www3.keizaireport.com/report.php/RID/467976/?rss	marketflash	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	脱非常時対応、はじめの一歩～ＥＣＢがPEPPの買い入れペースを減額：Europe Trends	http://www3.keizaireport.com/report.php/RID/467977/?rss	europetrends	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	ECBのラガルド総裁の記者会見～Moderately lower pace：井上哲也のReview on Central Banking	http://www3.keizaireport.com/report.php/RID/467978/?rss	moderatelylowerpace	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	要興業（東証二部）～東京23区を地盤に事業所からの廃棄物を扱う総合廃棄物処理業大手。外部要因が読みにくい状況は続くが、22年3月期利益は前期並みの会社計画：アナリストレポート	http://www3.keizaireport.com/report.php/RID/468000/?rss	廃棄物処理	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	フォースタートアップス（東証マザーズ）～スタートアップ企業を人材資源の側面から支援するための様々なサービスを展開。人材需要の回復により22年3月期は大幅な増収増益を見込む：アナリストレポート	http://www3.keizaireport.com/report.php/RID/468001/?rss	東証マザーズ	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	J-REITで加速する気候変動リスクへの対応	http://www3.keizaireport.com/report.php/RID/468006/?rss	jreit	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	地域金融強化に向けた取り組み～地域金融機関の経営基盤強化と日本銀行の施策：金融システムレポート別冊	http://www3.keizaireport.com/report.php/RID/468010/?rss	取り組み	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	ECB、資産購入ペースの減速を決定～ラガルド総裁はテーパリングではないと強調：マーケットレポート	http://www3.keizaireport.com/report.php/RID/468022/?rss	三井住友トラスト	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	ECB理事会（9月9日）の注目点～ECBは資産購入ペース減速決定、正常化議論は先送り：マーケット・レポート	http://www3.keizaireport.com/report.php/RID/468023/?rss	購入	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	金融市場NOW：2021年4～6月期世界貿易量過去最大の伸び～海運市況高騰を背景に海運株が堅調	http://www3.keizaireport.com/report.php/RID/468024/?rss	金融市場	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	9月ECB理事会 金融政策の現状維持を決定～PEPP資産購入ペース減速を決定：マーケットレポート	http://www3.keizaireport.com/report.php/RID/468025/?rss	現状維持	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	アジア・オセアニアリート市場は概ね上昇～経済正常化期待から堅調な推移へ	http://www3.keizaireport.com/report.php/RID/468026/?rss	三井住友	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	メジャーSQ通過後の日経平均株価の動きについて：市川レポート	http://www3.keizaireport.com/report.php/RID/468056/?rss	三井住友	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	FX Weekly（2021年9月10日号）～来週の相場見通し（1）ドル円：テーパリングより話題さらった自民党総裁選	http://www3.keizaireport.com/report.php/RID/468060/?rss	fxweekly	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	Weekly金融市場 2021年9月10日号～来週の注目材料、経済指標...	http://www3.keizaireport.com/report.php/RID/468070/?rss	weekly	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	ユーロはどこに行くのか～期待とリスク：林秀毅の欧州経済・金融リポート（最終回）	http://www3.keizaireport.com/report.php/RID/468071/?rss	日本経済研究センター	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	投資信託の購入で積立投資の利用が拡大～首都圏居住者を対象としたアンケート調査の結果から：農漁協・森組	http://www3.keizaireport.com/report.php/RID/468098/?rss	投資信託	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	週刊！投資環境（2021年9月10日号）～来週の注目点を皆さまにいち早くお届け...	http://www3.keizaireport.com/report.php/RID/468102/?rss	投資信託	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	【注目検索キーワード】心のバリアフリー	http://search.keizaireport.com/search.php/-/keyword=心のバリアフリー/?rss	検索キーワード	2021-09-11 00:00:00
金融	金融総合：経済レポート一覧	【お薦め書籍】もう価格で闘わない〜非価格経営を実現した24社の取り組み	https://www.amazon.co.jp/exec/obidos/ASIN/4866672781/keizaireport-22/	価格競争	2021-09-11 00:00:00
ニュース	@日本経済新聞 電子版	FX・暗号資産・iDeCo…子どもに聞かれて困る金融用語 https://t.co/rvF4fkMyNA	https://twitter.com/nikkei/statuses/1436477289366126595	金融用語	2021-09-10 23:50:48
ニュース	@日本経済新聞 電子版	米同時テロ20年、追悼始まる　バイデン氏「団結が強み」 https://t.co/W05ni5KTDn	https://twitter.com/nikkei/statuses/1436476815057424388	追悼	2021-09-10 23:48:55
ニュース	@日本経済新聞 電子版	独ビオンテック「5～11歳向けワクチン、近く申請」 https://t.co/mnkGikimIo	https://twitter.com/nikkei/statuses/1436475516182278146	近く	2021-09-10 23:43:46
ニュース	@日本経済新聞 電子版	けさ9月11日の日経電子版トップ（https://t.co/FsM9YUd2vm）3本です。 ▶障害者雇用、変革の「戦力」　29道府県が法定雇用率達成 https://t.co/rTVDKoxfGL ▶米地裁、Appleにアプ… https://t.co/TSm2qiG26A	https://twitter.com/nikkei/statuses/1436470831476092928	けさ月日の日経電子版トップ本です。	2021-09-10 23:25:09
海外ニュース	Japan Times latest articles	Japan Post to tie up with Sagawa Express on parcel deliveries	https://www.japantimes.co.jp/news/2021/09/11/business/japan-post-sagawa/	Japan Post to tie up with Sagawa Express on parcel deliveriesThe two firms said that they have reached a basic agreement to collaborate on parcel delivery operations to meet pandemic induced growth in delivery demand	2021-09-11 08:30:16
ニュース	BBC News - Home	All of Disney's 2021 movies to debut exclusively in cinemas	https://www.bbc.co.uk/news/world-us-canada-58524893?at_medium=RSS&at_campaign=KARANGA	cinemasall	2021-09-10 23:04:26
ニュース	BBC News - Home	Afghanistan: BBC catches up with Afghan journalist who fled Kabul	https://www.bbc.co.uk/news/world-asia-58523543?at_medium=RSS&at_campaign=KARANGA	taliban	2021-09-10 23:11:49
ニュース	BBC News - Home	Ros Atkins on... lorry driver shortages	https://www.bbc.co.uk/news/uk-58521211?at_medium=RSS&at_campaign=KARANGA	atkins	2021-09-10 23:11:32
ニュース	BBC News - Home	Covid: The Chelmsford bench telling women's stories of the pandemic	https://www.bbc.co.uk/news/uk-england-essex-58505014?at_medium=RSS&at_campaign=KARANGA	covid	2021-09-10 23:09:57
ニュース	BBC News - Home	Powerchair football: 'My hat-trick got Northern Ireland to the World Cup'	https://www.bbc.co.uk/news/uk-northern-ireland-58436616?at_medium=RSS&at_campaign=KARANGA	dystrophy	2021-09-10 23:09:46
ニュース	BBC News - Home	Week in pictures: 4-10 September 2021	https://www.bbc.co.uk/news/in-pictures-58501065?at_medium=RSS&at_campaign=KARANGA	images	2021-09-10 23:00:51
ニュース	BBC News - Home	'Vaccine passports make me even more reluctant to get a Covid jab'	https://www.bbc.co.uk/news/newsbeat-58505658?at_medium=RSS&at_campaign=KARANGA	covid	2021-09-10 23:01:26
ニュース	BBC News - Home	Covid: UK expats struggle with Australia lockdowns	https://www.bbc.co.uk/news/uk-england-hereford-worcester-58434251?at_medium=RSS&at_campaign=KARANGA	australia	2021-09-10 23:02:44
ニュース	BBC News - Home	Manic Street Preachers: 'A Design For Life saved us'	https://www.bbc.co.uk/news/entertainment-arts-58460903?at_medium=RSS&at_campaign=KARANGA	piano	2021-09-10 23:05:20
ニュース	BBC News - Home	Ethiopia: The country where a year lasts 13 months	https://www.bbc.co.uk/news/world-africa-57443424?at_medium=RSS&at_campaign=KARANGA	cultural	2021-09-10 23:44:27
ニュース	BBC News - Home	Can Raducanu end 44-year drought? Briton and fellow teenager Fernandez set for US Open final	https://www.bbc.co.uk/sport/tennis/58524261?at_medium=RSS&at_campaign=KARANGA	Can Raducanu end year drought Briton and fellow teenager Fernandez set for US Open finalEmma Raducanu seeks to become the first British woman to win a Grand Slam singles title in years when she takes on fellow teenager Leylah Fernandez in the US Open final	2021-09-10 23:26:27
ニュース	BBC News - Home	South American Premier League players set to avoid bans and be available for clubs	https://www.bbc.co.uk/sport/football/58522907?at_medium=RSS&at_campaign=KARANGA	South American Premier League players set to avoid bans and be available for clubsPremier League players are set to be cleared to compete this weekend after Brazil reportedly dropped their complaint about their failure to report for international duty	2021-09-10 23:21:22
ニュース	BBC News - Home	9/11 anniversary: Left without a parent after that day	https://www.bbc.co.uk/news/world-us-canada-58508260?at_medium=RSS&at_campaign=KARANGA	attacks	2021-09-10 23:50:04
LifeHuck	ライフハッカー［日本版］	庭の草花を使ってインテリアを充実させる方法	https://www.lifehacker.jp/2021/09/241309the-best-ways-to-use-your-outdoor-garden-to-decorate-yo.html	草花	2021-09-11 08:30:00
サブカルネタ	ラーブロ	2021/09/11 G1桐生ボート予想1~6R	http://feedproxy.google.com/~r/rablo/~3/4LvEhMEpmAc/single_feed.php	開催	2021-09-11 00:11:00
GCP	Cloud Blog	Scalable ML Workflows using PyTorch on Kubeflow Pipelines and Vertex Pipelines	https://cloud.google.com/blog/topics/developers-practitioners/scalable-ml-workflows-using-pytorch-kubeflow-pipelines-and-vertex-pipelines/	Scalable ML Workflows using PyTorch on Kubeflow Pipelines and Vertex PipelinesIntroductionML Ops is an ML engineering culture and practice that aims at unifying ML system development and ML system operation An important ML Ops design pattern is the ability to formalize ML workflows This allows them to be reproduced tracked and analyzed shared and more Pipelines frameworks support this pattern and are the backbone of an ML Ops story These frameworks help you to automate monitor and govern your ML systems by orchestrating your ML workflows  In this post we ll show examples of PyTorch based ML workflows on two pipelines frameworks OSS Kubeflow Pipelines part of the Kubeflow project and Vertex Pipelines We are also excited to share some new PyTorch components that have been added to the Kubeflow Pipelines repo  In addition we ll show how the Vertex Pipelines examples which require v of the KFP SDK can now also be run on an OSS Kubeflow Pipelines installation using the KFP v compatibility mode PyTorch on Google Cloud PlatformPyTorch continues to evolve rapidly with more complex ML workflows being deployed at scale Companies are using PyTorch in innovative ways for AI powered solutions ranging from autonomous driving to drug discovery surgical Intelligence and even agriculture MLOps and managing the end to end lifecycle for these real world solutions running at large scale continues to be a challenge  The recently launched Vertex AI is a unified ML Ops platform to help data scientists and ML engineers increase their rate of experimentation deploy models faster and manage models more effectively It brings AutoML and AI Platform together with some new ML Ops focused products into a unified API client library and user interface Google Cloud Platform and Vertex AI are a great fit for PyTorch with PyTorch support for Vertex AI training and serving and PyTorch based Deep Learning VM images and containers including PyTorch XLA support The rest of this post will show examples of PyTorch based ML workflows on two pipelines frameworks OSS Kubeflow Pipelines part of the Kubeflow project and Vertex Pipelines All the examples use the open source Python KFP Kubeflow Pipelines SDK which makes it straightforward to define and use PyTorch components Both pipelines frameworks provide sets of prebuilt components for ML related tasks support easy component pipeline step authoring and provide pipeline control flow like loops and conditionals automatically log metadata during pipeline execution support step execution caching and more Both of these frameworks make it straightforward to build and use PyTorch based pipeline components and to create and run PyTorch based workflows  Kubeflow PipelinesThe Kubeflow open source project includes Kubeflow Pipelines KFP a platform for building and deploying portable scalable machine learning ML workflows based on Docker containers The open source Kubeflow Pipelines backend runs on a Kubernetes cluster such as GKE Google s hosted Kubernetes You can install the KFP backend standalone ーvia CLI or via the GCP Marketplaceーif you don t need the other parts of Kubeflow  The OSS KFP examples highlighted in this post show several different workflows and include some newly contributed components now in the Kubeflow Pipelines GitHub repo These examples show how to leverage the underlying Kubernetes cluster for distributed training use a TensorBoard server for monitoring and profiling and more  Vertex PipelinesVertex Pipelines is part of Vertex AI and uses a different backend from open source KFP It is automated scalable serverless and cost effective you pay only for what you use Vertex Pipelines is the backbone of the Vertex AI ML Ops story and makes it easy to build and run ML workflows using any ML framework Because it is serverless and has seamless integration with GCP and Vertex AI tools and services you can focus on building and running your pipelines without dealing with infrastructure or cluster maintenance Vertex Pipelines automatically logs metadata to track artifacts lineage metrics and execution across your ML workflows and provides support for enterprise security controls like Cloud IAM VPC SC and CMEK The example Vertex pipelines highlighted in this post share some underlying PyTorch modules with the OSS KFP example and include use of the prebuilt Google Cloud Pipeline Components which make it easy to access Vertex AI services Vertex Pipelines requires v of the KFP SDK It is now possible to use the KFP v compatibility mode to run KFP V examples on an OSS KFP installation and we ll show how to do that as well PyTorch on Kubeflow Pipelines PyTorch KFP Components SDKIn collaboration across Google and Facebook we are announcing a number of technical contributions to enable large scale ML workflows on Kubeflow Pipelines with PyTorch This includes the PyTorch Kubeflow Pipelines components SDK with features for   Data loading and preprocessing Model Training using PyTorch Lightning as training loop Model profiling and visualizations using the new PyTorch Tensorboard Profiler Model deployment amp Serving using TorchServe KFServing with canary rollouts autoscaling and Prometheus monitoring Model Interpretability using CaptumDistributed training using the PyTorch job operator for KFPHyperparameter tuning using Ax BoTorchML Metadata for Artifact Lineage Tracking Cloud agnostic artifacts storage component using Minio Computer Vision and NLP workflows are available for Open Source Kubeflow Pipelines deployed on any cloud or on prem Google Cloud Vertex AI Pipelines for Serverless pipelines solutionFigure NLP BERT Workflow on Open Source KFP with PyTorch profiler and Captum insights top left Pipeline View top right PyTorch Tensorboard Profiler for the training node bottom Captum model insights for the model prediction Start by setting up a KFP cluster with all the prerequisites and then follow one of the examples under the pytorch samples here Sample notebooks and full pipelines examples are available for the following  Computer Vision CIFAR pipeline basic notebook and notebook with Captum InsightsNLP BERT pipeline and notebook with Captum for model interpretability Distributed training sample using the PyTorch job operatorHyperparameter optimization sample using Ax BotorchNote All the samples are expected to run both on prem and on any cloud using CPU or GPUs for training and inference Minio is used as the cloud agnostic storage solution A custom TensorBoard image is used for viewing the PyTorch Profiler PyTorch on Kubeflow Pipelines BERT NLP exampleLet s do a walkthrough of the BERT example notebook  Training the PyTorch NLP modelOne starts by defining the KFP pipeline with all the tasks to execute The tasks are defined using the component yamls with configurable parameters All templates are available here The training component takes as input a PyTorch Lightning script along with the input data and parameters and returns the model checkpoint tensorboard profiler traces and the metadata for metrics like confusion matrix and artifacts tracking If you are using GPUs for training set the gpus to value gt and use ddp as the default accelerator type You will also need to specify the gpu limit and node selector constraint for the cluster  For generating traces for the PyTorch Tensorboard profiler “profiler pytorch is set in script args The confusion matrix gets logged as part of the ML metadata in the KFP artifacts store along with all the inputs and outputs and the detailed logs for pipeline run You can view these from the pipeline graph and the lineage explorer as shown in Figure below Caching is enabled by default so if you run the same pipeline again with the same inputs the results will be picked up from the KFP cache Figure Pipeline graph view Visualization for Confusion Matrix and ML Metadata in the Lineage ExplorerThe template mapping json config file is used for generating the component yaml files from the templates and setting the script names and docker container with all the code You can create a similar Docker container for your own pipeline  Debugging using PyTorch Tensorboard ProfilerThe PyTorch Tensorboard Profiler provides insights into the performance bottlenecks like inefficiency for loading data underutilization of the GPUs SM efficiency and CPU GPU thrashing and is very helpful for debugging performance issues Check out the Profiler blog for the latest updates  In the KFP pipeline the Tensorboard Visualization component handles all the magic of making the traces available to the PyTorch Tensorboard profiler therefore it is created before starting the training run  The profiler traces are saved in the tensorboard logs bucket under the pipeline run ID and are available for viewing after the training step completes You can access TensorBoard from the Visualization component of the pipeline after clicking the “Start Tensorboard button Full traces are available from the PyTorch Profiler view in the Tensboard as shown below Figure PyTorch Profiler Trace viewA custom docker container is used for the PyTorch profiler plugin and you can specify the image name by setting the TENSORBOARD IMAGE parameter  Model Serving using KFServing with TorchServePyTorch model serving for running the predictions is done via the KFServing TorchServe integration It supports prediction and explanation APIs canary rollouts with autoscaling and monitoring using Prometheus and Grafana  For the NLP BERT model the bert handler py defines the TorchServe custom handler with logic for loading the model running predictions and doing the pre processing and post processing  The training component generates the model files as a model archiver package and this gets deployed onto TorchServe The minio op is used for making the model archiver and the TorchServe config properties available to the deployment op For deploying the model you simply need to set the KFServing Inference yaml with the relevant values e g for the GPU inference you will pass the model storage location and the number of GPUs Using Captum for Model InterpretabilityCaptum ai is the Model Interpretability library for PyTorch In the NLP example we use the explanation API of KFserving and TorchServe to get the model insights for interpretability The explain handler defines the IntegratedGradient computation logic which gets called via the explain endpoint and returns a json response with the interpretability output The results are rendered in the notebook using Captum Insights This renders the color coded visualization for the word importance Distributed training using PyTorch job operatorThe Kubeflow PyTorch job operator is used for distributed training and it takes as inputs the job spec for the master and worker nodes along with the option to customize other parameters via the pytorch launcher component PyTorch on Kubeflow Pipelines CIFAR HPO exampleHyperparameter optimization using Ax BoTorchAx is the adaptive experimentation platform for PyTorch and BoTorch is the Bayesian Optimization library They are used together for Hyperparameter optimization  The CIFAR HPO notebook describes the usage for this We start off by generating the experiment trials with the parameters that we want to optimize using the ax generate trials component Next the trials are run in parallel using the ax train component And finally the ax complete trials componentis used for processing the results for the best parameters from the Hyperparameter search The best parameters can be viewed under Input Output section of ax complete trials as shown in the figure below PyTorch on Vertex Pipelines CIFAR image classification exampleThe Vertex Pipelines examples in this post also use the KFP SDK and include use of the Google Cloud Pipeline Components which support easy access to Vertex AI services Vertex Pipelines requires v of the KFP SDK So these examples diverge from the OSS KFP v based examples above though the components share some of the same data processing and training base classes It is now possible to use the KFP v compatibility mode to run KFP V examples on an OSS KFP installation and we ll show how to do that as well An example PyTorch Vertex Pipelines notebook shows two variants of a pipeline that do data preprocessing train a PyTorch CIFAR resnet model convert the model to archive format build a torchserve serving container upload the model container configured for Vertex AI custom prediction and deploy the model serving container to an endpoint so that it can serve prediction requests on Vertex AI  In the example the torchserve serving container is configured to use the kfserving service envelope which is compatible with the Vertex AI prediction service Training the PyTorch image classification modelThe difference between the two pipeline variants in the notebook is in the training step One variant does on step node single GPU trainingーthat is it runs the training job directly on the Vertex pipeline step node We can specify how the pipeline step instance is configured to give the node instance the necessary resources This fragment from the KFP pipeline definition shows that configuration which specifies to use one Nvidia V for the training step in the pipeline The other example variant in the notebook shows multi GPU single node training via Vertex AI s support for custom training using the Vertex AI SDK  From the custom training pipeline step a custom job is defined passing the URI of the container image for the PyTorch training code Then the custom training job is run specifying machine and accelerator types and number of accelerators PyTorch prebuilt training containers are available as well though for this example we used PyTorch v which at time of writing is not yet available in the prebuilt set Defining KFP PipelinesSome steps in the example KFP v pipelines are built from Python function based custom componentsーthese make it easy to develop pipelines interactively and are defined right in the example notebookーand other steps are defined using a set of prebuilt components that make it easy to interact with Vertex AI and other servicesーthe steps that upload the model create an endpoint and deploy the model to the endpoint The custom components include pipeline steps to create a model archive from the trained PyTorch model and the model file and to build a torchserve container image using the model archive file and the serving config properties The torchserve build step uses Cloud Build to create the container image These pipeline component definitions can be compiled to yaml files as shown in the example notebook The yaml component definitions are portable they can be placed under version control and shared and used to create pipeline steps for use in other pipeline definitions The KFP pipeline definition looks like the following with some detail removed See the notebook for the full definition Some pipeline steps consume as inputs the outputs of other steps The prebuilt google cloud pipeline components make it straightforward to access Vertex AI services Note that the ModelDeployOp step is configured to serve the trained model on a GPU instance Here s the pipeline graph for one of the Vertex Pipelines examples The pipeline graph for one of the KFP v example pipelines running on Vertex PipelinesAs a pipeline runs metadata about the run including its Artifacts executions and events is automatically logged to the Vertex ML Metadata server The Pipelines Lineage Tracker part of the UI uses the logged metadata to render an Artifact centric view of pipeline runs showing how Artifacts are connected by step executions In this view it s easy to track where multiple pipeline runs have used the same artifact Where a pipeline is able to leverage caching you will often notice that multiple pipeline runs are able to use the same cached step outputs Vertex Pipeline artifact lineage tracking Using KFP v compatibility mode to run the pipelines on an OSS KFP installationIt is now possible to run the same KFP v pipelines in the Vertex example above on an OSS KFP installation  Kubeflow Pipelines SDK v compatibility mode lets you use the new pipeline semantics in v and gain the benefits of logging your metadata to ML Metadata Compatibility mode means that you can develop a pipeline on one platform and run it on the other Here is the pipeline graph for the same pipeline shown above running on Vertex Pipelines but running on an OSS KFP installation  If you compare it to the Vertex Pipelines graph in the figure above you can see that they have the same structure The example s README gives more information about how to do the installation and the example PyTorch Vertex Pipelines notebook includes sections that show how to launch an OSS KFP pipeline run once you ve done the setup The pipeline graph for one of the KFP v example pipelines running on an OSS KFP installation Next stepsThis post showed some examples of how to build scalable ML workflows using PyTorch running on both OSS Kubeflow Pipelines and Vertex Pipelines  Kubeflow and Vertex AI make it easy to use PyTorch on GCP and we have announced some new PyTorch KFP components that make creating PyTorch based ML workflows even easier We also showed how the Vertex Pipelines examples which require v of the KFP SDK can now also be run on an OSS Kubeflow Pipelines installation using the KFP v compatibility mode Please check out the samples here and here and let us know what you think You can provide feedback on the PyTorch Forums or file issues on the Kubeflow Pipelines Github repository AcknowledgementsThe authors would like to thank the contributions from the following people for making this work possible Pavel Dournov Henry Tappen Yuan Gong Jagadeesh Jaganathan Srinath Suresh Alexey Volkov Karl Weinmeister Vaibhav Singh and the Vertex Pipelines team Related ArticlePyTorch on Google Cloud How To train and tune PyTorch models on Vertex AIWith the PyTorch on Google Cloud blog series we will share how to build train and deploy PyTorch models at scale how to create reprodu Read Article	2021-09-10 23:30:00