TECH |
Engadget Japanese |
Googleアシスタントからの音声入力で電源オフ、Android 12で実装の噂 |
https://japanese.engadget.com/google-assistant-power-off-073013073.html
|
togoogle |
2021-05-25 07:30:13 |
TECH |
Engadget Japanese |
刻印がちゃんと見えるSimplismのAirTag用シリコンケースを試してみた |
https://japanese.engadget.com/air-tag-simplism-070018437.html
|
airtag |
2021-05-25 07:00:18 |
ROBOT |
ロボスタ |
東大発AIベンチャーが映像内の個人情報を自動でマスキングする「モザイクAI」のリアルタイム処理を実現 |
https://robotstart.info/2021/05/25/smith-motors-masking-ai.html
|
交通事故 |
2021-05-25 07:38:02 |
ROBOT |
ロボスタ |
Amazon Echoで睡眠の質を向上させる3つのテクニックを紹介 今すぐ改善できる2つのポイントは「五感」と「規則正しいリズム」 |
https://robotstart.info/2021/05/25/alexa-good-sleep-technique.html
|
alexa |
2021-05-25 07:17:26 |
IT |
ITmedia 総合記事一覧 |
[ITmedia PC USER] macOS Big Surが11.4にアップデート 多数の脆弱性と不具合が修正 |
https://www.itmedia.co.jp/pcuser/articles/2105/25/news140.html
|
apple |
2021-05-25 16:34:00 |
IT |
ITmedia 総合記事一覧 |
[ITmedia ビジネスオンライン] 移動中の会議を快適に DNPや日産が「移動会議室」の実証実験 |
https://www.itmedia.co.jp/business/articles/2105/25/news134.html
|
itmedia |
2021-05-25 16:25:00 |
IT |
MOONGIFT |
Paperboard - WebでもCUIでも使えるブックマーク/リーディングリスト管理 |
http://feedproxy.google.com/~r/moongift/~3/JU3jiS_fsvE/
|
PaperboardWebでもCUIでも使えるブックマークリーディングリスト管理Webブラウジングをしていて、後で読みたいと思うページに出会うことがあります。 |
2021-05-25 17:00:00 |
IT |
情報システムリーダーのためのIT情報専門サイト IT Leaders |
AIが発注量を算出して在庫切れや過剰発注を防止、infonervの「α-発注」 | IT Leaders |
https://it.impress.co.jp/articles/-/21534
|
AIが発注量を算出して在庫切れや過剰発注を防止、infonervの「α発注」ITLeadersinfonervは年月日、小売・卸業に向けて、適切な発注量をAIで自動的に算出するツール「α発注」を発表した。 |
2021-05-25 16:13:00 |
python |
Pythonタグが付けられた新着投稿 - Qiita |
NumPyとOpenCVでエラーが出る件 |
https://qiita.com/thinknee/items/9489119c8b85d9323132
|
|
2021-05-25 16:41:12 |
python |
Pythonタグが付けられた新着投稿 - Qiita |
【Python】クラス変数の妙 |
https://qiita.com/dsuzu88/items/a80c1a74cd9337751f94
|
【Python】クラス変数の妙はじめにPythonを勉強していて、妙を感じるところがあったので備忘録として残そうと思いました。 |
2021-05-25 16:40:25 |
python |
Pythonタグが付けられた新着投稿 - Qiita |
【完結編!】SDF EditorでShadertoyやGLSLのモデリングが劇的に簡単に! |
https://qiita.com/quittardis/items/488ec98983367ce1bd78
|
これをマウス右ボタンgtShadertoyCreateProtbaleGLSLPreviewで得られるviewSDFhtmlはローカルでもweb上でも同じRaymarchingのリアルタイムCGが表示できる。 |
2021-05-25 16:38:52 |
js |
JavaScriptタグが付けられた新着投稿 - Qiita |
【JS】includes()で特定要素を複数指定したいとき |
https://qiita.com/mtoutside/items/e34e06c674a8e6623ba4
|
Arrayprototypeincludesincludesメソッドは、特定の要素が配列に含まれているかどうかをtrueまたはfalseで返します。 |
2021-05-25 16:45:06 |
Program |
[全てのタグ]の新着質問一覧|teratail(テラテイル) |
c言語ファイルの足し算 |
https://teratail.com/questions/340279?rss=all
|
|
2021-05-25 16:55:27 |
Program |
[全てのタグ]の新着質問一覧|teratail(テラテイル) |
XpythonのXGBoostで信頼区間 |
https://teratail.com/questions/340278?rss=all
|
xgboost |
2021-05-25 16:48:15 |
Program |
[全てのタグ]の新着質問一覧|teratail(テラテイル) |
Herokuエラー Failed to install gems via Bundler. |
https://teratail.com/questions/340277?rss=all
|
|
2021-05-25 16:44:24 |
Program |
[全てのタグ]の新着質問一覧|teratail(テラテイル) |
Swiperについてです |
https://teratail.com/questions/340276?rss=all
|
SwiperについてですltscriptgtletnbspmySwipernbspnbspnewnbspSwipernbspaposswipercontaineraposnbsploopnbsptruepaginationnbspelnbspaposswiperpaginationaposclickablenbsptruenavigationnbspnextElnbspaposswiperbuttonnextaposprevElnbspaposswiperbuttonprevaposltscriptgtエラーがでて動きません。 |
2021-05-25 16:43:29 |
Program |
[全てのタグ]の新着質問一覧|teratail(テラテイル) |
横幅指定した幅を100%で表示できるのか知りたい |
https://teratail.com/questions/340275?rss=all
|
anameviewportcontentwidth |
2021-05-25 16:37:43 |
Program |
[全てのタグ]の新着質問一覧|teratail(テラテイル) |
passwordにだけallow_nilをつける理由 |
https://teratail.com/questions/340274?rss=all
|
passwordにだけallownilをつける理由わからないことRailsチュートリアルでの話です。 |
2021-05-25 16:35:42 |
Program |
[全てのタグ]の新着質問一覧|teratail(テラテイル) |
拾うスクリプトを作っているのですが、 |
https://teratail.com/questions/340273?rss=all
|
|
2021-05-25 16:30:03 |
Program |
[全てのタグ]の新着質問一覧|teratail(テラテイル) |
「.0」を明示した数値→文字列変換 |
https://teratail.com/questions/340272?rss=all
|
「」を明示した数値→文字列変換float型変数から書式の決まった文字列を作成したい。 |
2021-05-25 16:04:09 |
Ruby |
Rubyタグが付けられた新着投稿 - Qiita |
【Rails】留学向けQ&Aサイト『Answer-ly』(ポートフォリオ)� |
https://qiita.com/tani__san929/items/cae0adb9b691a223382f
|
最終的には英語力は大幅にUPしました笑上記の自身の留学体験談から、私は、同じような不安を持つ人達を助けたい、不安を取り除き、より良い留学生活にしてほしいという思いと、つの不足情報不足機会不足を解消したく、今回のPFの留学向けQampAサイトを製作することに決めました。 |
2021-05-25 16:07:43 |
AWS |
AWSタグが付けられた新着投稿 - Qiita |
AWS AMIの作成からアカウント間での共有方法まで |
https://qiita.com/mtanakadesu/items/47738ab2116c8404911b
|
イメージを作成にて、イメージ名任意の値を入力し、「イメージの作成」をクリックしその後AMIが作成され、ステータスが「available」状態になることを確認します。 |
2021-05-25 16:44:54 |
AWS |
AWSタグが付けられた新着投稿 - Qiita |
[MAC]aws-cliでiamアカウントの接続設定 |
https://qiita.com/y-matsumoto/items/e2322c116147ff7f1abb
|
credentials に は アクセス キー と シークレットキー 。 |
2021-05-25 16:27:25 |
AWS |
AWSタグが付けられた新着投稿 - Qiita |
【Rails】留学向けQ&Aサイト『Answer-ly』(ポートフォリオ)� |
https://qiita.com/tani__san929/items/cae0adb9b691a223382f
|
最終的には英語力は大幅にUPしました笑上記の自身の留学体験談から、私は、同じような不安を持つ人達を助けたい、不安を取り除き、より良い留学生活にしてほしいという思いと、つの不足情報不足機会不足を解消したく、今回のPFの留学向けQampAサイトを製作することに決めました。 |
2021-05-25 16:07:43 |
AWS |
AWSタグが付けられた新着投稿 - Qiita |
terraformでコンテナに機密情報を渡す |
https://qiita.com/ms2geki/items/08853c86d49863e6a7eb
|
terraformでコンテナに機密情報を渡すECSでコンテナに機密情報を渡す方法タスク定義のコンテナ編集の環境変数メニューでValueFromにssmパラメーターのarnを指定し、より安全にコンテナに機密情報を渡すことが可能です。 |
2021-05-25 16:04:13 |
Docker |
dockerタグが付けられた新着投稿 - Qiita |
terraformでコンテナに機密情報を渡す |
https://qiita.com/ms2geki/items/08853c86d49863e6a7eb
|
terraformでコンテナに機密情報を渡すECSでコンテナに機密情報を渡す方法タスク定義のコンテナ編集の環境変数メニューでValueFromにssmパラメーターのarnを指定し、より安全にコンテナに機密情報を渡すことが可能です。 |
2021-05-25 16:04:13 |
Ruby |
Railsタグが付けられた新着投稿 - Qiita |
ActiveRecord::Base.connection.column_exists? メソッドのユースケース |
https://qiita.com/kk-icare/items/ac31c02090c04bc98382
|
ActiveRecordBaseconnectioncolumnexistsメソッドのユースケースはじめに今回はActiveRecordBaseconnectionのcolumnexistsメソッドの内部実装とユースケースについて書いてみたいと思います。 |
2021-05-25 16:29:08 |
Ruby |
Railsタグが付けられた新着投稿 - Qiita |
【Rails】留学向けQ&Aサイト『Answer-ly』(ポートフォリオ)� |
https://qiita.com/tani__san929/items/cae0adb9b691a223382f
|
最終的には英語力は大幅にUPしました笑上記の自身の留学体験談から、私は、同じような不安を持つ人達を助けたい、不安を取り除き、より良い留学生活にしてほしいという思いと、つの不足情報不足機会不足を解消したく、今回のPFの留学向けQampAサイトを製作することに決めました。 |
2021-05-25 16:07:43 |
技術ブログ |
Developers.IO |
ALBで静的IPアドレスを使うためのテンプレート(NLB-ALB-EC2構成)からNLB-EC2構成に切り替えてみた |
https://dev.classmethod.jp/articles/nlb-alb-ec2-to-nlb-ec2/
|
application |
2021-05-25 07:30:28 |
海外TECH |
DEV Community |
CSS ::placeholder and :placeholder-shown |
https://dev.to/suprabhasupi/css-placeholder-and-placeholder-shown-2cpk
|
CSS placeholder and placeholder shownThe placeholder shown CSS pseudo class represents any or element that is currently displaying placeholder text lt input type text placeholder some text gt input placeholder shown font family Baloo cursive font weight bold background color pink Output In above snippet If user has not typed anything then placeholder shown background color will be pinkWhen user has typed something then there will be no placeholder shown and the background color will be transparentNOTE placeholder shown won t work if there will be no placeholder textExample lt input type text gt input placeholder shown background color pink The above snippet doesn t work placeholder shown vs placeholder We can use placeholder shown to style the input element input placeholder shown border px solid pink background yellow color blue Here we set color blue but it didn t work That s because placeholder shown will only target the input itself But for the actual placeholder text you have to use the pseudo element placeholder placeholder pseudo element can be used to style the placeholder text in lt input gt or lt textarea gt element input placeholder color blue Output If you wanted to add styles for empty placeholder we can pass empty string as placeholder lt input type text placeholder gt Output Now why don t we combine two selector not and placeholder shown So we can target if the input is not empty then we can apply css lt input placeholder some text value Check with empty gt input not placeholder shown background color pink Output In above snippet If the input has value then the background color will get appliedIf the input is empty then background color not appliesNOTE placeholder does not work with not selector Reference MDN Doc PlaceholderMDN Doc placeholder showncaniuse placeholder shown Twitter suprabha me Instagram |
2021-05-25 07:16:30 |
海外TECH |
DEV Community |
Build a COVID-19 Tracker Application using HTML,CSS and JavaScript [interesting features included] |
https://dev.to/thenerdydev/build-a-covid-19-tracker-application-using-html-css-and-javascript-interesting-features-included-23ld
|
Build a COVID Tracker Application using HTML CSS and JavaScript interesting features included Hey everyone In this article I will show you how you can make a COVID Tracker Application with interesting features To learn how you can create one check the video provided down below You can also have a look at the live link of the project that we make in this video Images FeaturesClustering of points based on the countries and each cluster will represent a set of locations within it that are prone to COVID Reverse Geocoding using Latitude and Longitude to get the place name and getting the number of deaths and number of confirmed cases for each location that is present within the country Live Link Learn how to make one with same features in this video PS I am working on a BRAND new FREE Web Developer Full Course on YouTube check the below article to know more about the topics that we cover in this course Web Developer Full Course HTML CSS JavaScript Node js and MongoDB The Nerdy Dev・Apr ・ min read html css node javascript Follow me on Twitter Follow me on Twitter YouTube |
2021-05-25 07:07:43 |
海外TECH |
DEV Community |
Understanding Promise.all, Promise.race and Promise.allSettled |
https://dev.to/thenerdydev/understanding-promise-all-promise-race-and-promise-allsettled-2g1g
|
Understanding Promise all Promise race and Promise allSettledHey everyone In this article let us discuss some very important methods that we have in the Promise API Promise all Promise allSettled Promise race So let s start with Promise all Promise allThe Promise all method takes an iterable of promises as an input and returns a single Promise that resolves to an array of the results of the input promises This returned promise will resolve when all of the input s promises have resolved or if the input iterable contains no promises It rejects immediately upon any of the input promises rejecting or non promises throwing an error and will reject with this first rejection message error Promise allSettledThe Promise allSettled method returns a promise that resolves after all of the given promises have either fulfilled or rejected with an array of objects that each describes the outcome of each promise It is typically used when you have multiple asynchronous tasks that are not dependent on one another to complete successfully or you d always like to know the result of each promise In comparison the Promise returned by Promise all may be more appropriate if the tasks are dependent on each other if you d like to immediately reject upon any of them rejecting Promise raceThe Promise race method returns a promise that fulfills or rejects as soon as one of the promises in an iterable fulfills or rejects with the value or reason from that promise So this is it for this article I do have a video that covers the same with a lot of interesting examples so feel free to check that If you are looking to learn Web Development I have curated a FREE course for you on my YouTube Channel check the below article Web Developer Full Course HTML CSS JavaScript Node js and MongoDB The Nerdy Dev・Apr ・ min read html css node javascript If you have spare hours then utilize them by creating these JavaScript Projects in under Hours JavaScript Projects in Hours The Nerdy Dev・Oct ・ min read javascript webdev Follow me on Twitter Check out my YouTube Channel |
2021-05-25 07:01:41 |
Apple |
AppleInsider - Frontpage News |
WaterField launches new leather keychain, luggage tag for AirTag |
https://appleinsider.com/articles/21/05/25/waterfield-launches-new-leather-keychain-luggage-tag-for-airtag?utm_medium=rss
|
WaterField launches new leather keychain luggage tag for AirTagWaterField Designs has launched a new leather keychain and luggage tag for Apple s AirTag tracking accessory Credit WaterField DesignsThe new AirTag accessories feature designs crowdsourced from more than customers Crafted from full grain leather and available in multiple colors WaterField says they ll protect the AirTag from scratches and hide them from potential thieves Read more |
2021-05-25 07:45:14 |
医療系 |
医療介護 CBnews |
【感染症情報】プール熱の患者報告数が倍増-RSウイルス感染症は18年以降の同期比で最多 |
https://www.cbnews.jp/news/entry/20210525160230
|
医療機関 |
2021-05-25 16:20:00 |
金融 |
ニッセイ基礎研究所 |
保険会社の気候変動リスクへの対応(欧州)-EIOPAの意見書の紹介 |
https://www.nli-research.co.jp/topics_detail1/id=67845?site=nli
|
目次ーはじめにー気候変動問題への世界全体の取り組み状況の概説気候変動の概要気候変動問題人為的なものへの、国連気候変動枠組み条約締結国会議COPの動きー保険・年金関係における対応策など今回紹介するEIOPAの意見書EIOPAの基本的な方針気候変動がもたらすと考えられるリスクの種類今後保険監督者が保険会社に期待要求する対応ーおわりに「ESG」のファクターである気候変動問題については、国際社会が一体となって直ちに取り組むべき重要な課題であるとされている。 |
2021-05-25 16:59:28 |
金融 |
ニッセイ基礎研究所 |
今週のレポート・コラムまとめ【5/18~5/24】:雇用関連統計21年3月-雇用情勢は持ち直すも、対面型サービス業は依然厳しい |
https://www.nli-research.co.jp/topics_detail1/id=67844?site=nli
|
今週のレポート・コラムまとめ【】雇用関連統計年月ー雇用情勢は持ち直すも、対面型サービス業は依然厳しいNo日本株の見通し年後半は調整必至かnbspNoコロナ禍における生活の変化ー「新型コロナによる暮らしの変化に関する調査」からみる生活不安の変化と地域間較差nbspNoJリート保有物件の価格はコロナ禍でも下落せずnbspNo米国株式インデックス投資、どれを選べば良いのかーSPvsダウ平均vsナスダックNo年度介護報酬改定を読み解くー難しい人材不足への対応、科学化や予防重視の利害得失を考える研究員の眼nbspドリームジャンボの期待ー宝くじのポートフォリオをどう組成するnbsp中国、インフラ公募REIT上場へ計銘柄、発行総額は千億円超となる見通しnbsp原油高が企業収益に与える影響nbsp「三角関数」と「波」の関係ー三角関数による「波」の表現と各種の波電磁波、音波、地震波等ーnbspーWeeklyエコノミスト・レターnbsp・年度経済見通し年月nbspQE速報月期の実質GDPは前期比年率ー緊急事態宣言の影響で四半期ぶりのマイナス成長nbspー基礎研マンスリーnbspドル円円回復への距離感マーケット・カルテ月号nbspー不動産投資レポートnbspわが国の不動産投資市場規模商業施設の「収益不動産」は約兆円、物流施設は約兆円、ホテル・旅館は約兆円。 |
2021-05-25 16:29:47 |
ニュース |
ジェトロ ビジネスニュース(通商弘報) |
2021年の貿易見通しを上方修正、GDP成長率見通しは維持 |
https://www.jetro.go.jp/biznews/2021/05/4d0aa6fb353152e9.html
|
上方修正 |
2021-05-25 07:10:00 |
海外ニュース |
Japan Times latest articles |
Two months before Olympics, U.S. advises against travel to Japan |
https://www.japantimes.co.jp/news/2021/05/25/national/us-japan-travel-advisory/
|
Two months before Olympics U S advises against travel to Japan“Necessary travel has not been prohibited Foreign Minister Toshimitsu Motegi said Tuesday as officials maintained the sporting event would not be affected |
2021-05-25 16:51:19 |
海外ニュース |
Japan Times latest articles |
Uniqlo made a bet on comfortable bras. Now it’s paying off. |
https://www.japantimes.co.jp/news/2021/05/25/business/corporate-business/uniqlo-bra-sales/
|
japan |
2021-05-25 16:35:25 |
海外ニュース |
Japan Times latest articles |
Japan to impose 10-day quarantine after travel to India, five other countries |
https://www.japantimes.co.jp/news/2021/05/25/national/india-10-day-quarantine/
|
lanka |
2021-05-25 16:05:36 |
ニュース |
BBC News - Home |
Covid: Eight Covid areas should avoid indoor gatherings |
https://www.bbc.co.uk/news/uk-england-57232728
|
indian |
2021-05-25 07:03:39 |
ニュース |
BBC News - Home |
Government borrowing fell in April as UK reopened |
https://www.bbc.co.uk/news/business-57237574
|
april |
2021-05-25 07:08:07 |
ニュース |
BBC News - Home |
Sasha Johnson: Black Lives Matter activist shot by group of men |
https://www.bbc.co.uk/news/uk-england-london-57238301
|
johnson |
2021-05-25 07:37:11 |
ニュース |
BBC News - Home |
Nadia Whittome: Labour MP takes leave of absence with PTSD |
https://www.bbc.co.uk/news/uk-england-nottinghamshire-57232026
|
health |
2021-05-25 07:01:08 |
LifeHuck |
ライフハッカー[日本版] |
サブバックが着脱可能&抜群の収納力「3WAYバックパック」終了間近 |
https://www.lifehacker.jp/2021/05/234841-machi-ya-3waybagpack-end.html
|
折りたたみ |
2021-05-25 16:30:00 |
GCP |
Google Cloud Platform Japan 公式ブログ |
Google Workspace の活用でコラボレーションを変革 |
https://cloud.google.com/blog/ja/products/workspace/next-evolution-of-collaboration-for-google-workspace/
|
本日より、チームですでに作業しているGoogleドキュメント、Googleスプレッドシート、スライドから直接、ウェブ上のGoogleMeetの通話にコンテンツを表示する機能を展開します。 |
2021-05-25 09:00:00 |
北海道 |
北海道新聞 |
ファイザー製ワクチン接種年齢引き下げ 厚労省、28日に審議 |
https://www.hokkaido-np.co.jp/article/547695/
|
厚生労働相 |
2021-05-25 16:12:43 |
北海道 |
北海道新聞 |
直球走り駒苫から7奪三振 東海大札幌高・エース門別 |
https://www.hokkaido-np.co.jp/article/547493/
|
北海道新聞社 |
2021-05-25 16:11:52 |
北海道 |
北海道新聞 |
東証続伸、終値189円高2万8553円 2週間ぶり高値、仮想通貨安定で |
https://www.hokkaido-np.co.jp/article/547755/
|
仮想通貨 |
2021-05-25 16:11:04 |
ニュース |
Newsweek |
東京都25日のコロナ新規感染542人 重症者71人に増加 |
https://www.newsweekjapan.jp/stories/world/2021/05/25400.php
|
|
2021-05-25 16:45:00 |
ニュース |
Newsweek |
外国人を犯罪者予備軍とみなす日本の入管の許されざる実態 |
https://www.newsweekjapan.jp/stories/world/2021/05/post-96359.php
|
|
2021-05-25 16:45:00 |
ニュース |
Newsweek |
マスク不要のCDC勧告、米でも異論噴出 ワクチン未接種でもマスク不要との誤解も |
https://www.newsweekjapan.jp/stories/world/2021/05/cdc-17.php
|
しかし、今回マスク不要がアナウンスされたことで、一般市民の間では歓迎する人々がいる反面、混乱も起きているという。 |
2021-05-25 16:30:37 |
IT |
週刊アスキー |
新作Switch用ソフト『スサノオ~日本神話RPG~』が7月22日に発売決定! |
https://weekly.ascii.jp/elem/000/004/056/4056334/
|
nintendo |
2021-05-25 16:55:00 |
IT |
週刊アスキー |
レンジでらくらくの冷凍ポテトが「ブタメン」とコラボ! ブタメンパウダーでおつまみにも良さそう |
https://weekly.ascii.jp/elem/000/004/056/4056195/
|
冷凍ポテト |
2021-05-25 16:30:00 |
IT |
週刊アスキー |
シリーズ初!『真・女神転生III NOCTURNE HD REMASTER』がSteamで全世界同時配信開始! |
https://weekly.ascii.jp/elem/000/004/056/4056331/
|
inocturnehdremaster |
2021-05-25 16:25:00 |
IT |
週刊アスキー |
コクヨ、B5サイズのプリントがそのまま貼れる「キャンパスノート(プリント貼付用)」6月9日リニューアル発売 |
https://weekly.ascii.jp/elem/000/004/056/4056311/
|
追加 |
2021-05-25 16:20:00 |
IT |
週刊アスキー |
話題のスイーツ「マリトッツォ」をアレンジ! 横浜ロイヤルパークホテルが6月1日から2種販売 |
https://weekly.ascii.jp/elem/000/004/056/4056303/
|
横浜ロイヤルパークホテル |
2021-05-25 16:15:00 |
IT |
週刊アスキー |
戸塚の歴史と風景をイメージしたプレートはいかが? TSUBAKI食堂とコラボしたこまちカフェ「夏の畑プレート」6月から提供 |
https://weekly.ascii.jp/elem/000/004/056/4056300/
|
tsubaki |
2021-05-25 16:10:00 |
GCP |
Cloud Blog |
3x Dataflow Throughput with Auto Sharding for BigQuery |
https://cloud.google.com/blog/products/data-analytics/3x-dataflow-throughput-auto-sharding-bigquery/
|
x Dataflow Throughput with Auto Sharding for BigQueryMany of you rely on Dataflow to build and operate mission critical streaming analytics pipelines A key goal for us the Dataflow team is to make the technology work for users rather than the other way around Autotuning as a fundamental value proposition Dataflow offers is a key part of making that goal a reality it helps you focus on your use cases by eliminating the cost and burden of having to constantly tune and re tune your applications as circumstances change Dataflow already includes many autotuning capabilities and we are excited to announce a new capability today that further simplifies streaming data pipelines Google BigQuery is one of the most common storage systems that Dataflow Streaming pipelines export results to The Beam SDKs include built in transforms known as BigQuery I O connector that can read data from and write data to BigQuery tables While the write transform has been optimized to take advantage of various BigQuery capabilities e g dynamic destinations currently Dataflow relies on users to manually configure numFileShards or numStreamingKeys to parallelize BigQuery writes This is painful for customers because it is extremely challenging to get the configuration right and requires a lot of manual efforts to maintain it Current ChallengesManually selecting an optimal number of shards is a very hard problem and results in following issues Too low a sharding value leads to insufficient execution parallelism and therefore limited throughput Too high a sharding value leads to increased overhead both on the Dataflow side and BigQuery side which risks diminishing returns In some cases pipeline throughput drops further if it exceeds BigQuery quotas and limits due to high sharding The fundamental issue here is that one size does not fit all Sharding for one shape of the data may not be the right choice when circumstances change e g when the volume changes or when there are “hot spots with much more volume than the others The following illustration shows what happens with manual fixed sharding when streaming data to a BigQuery table As you can see irrespective of the volume of data the number of shards remains constant and becomes insufficient when the input rate bumps As a result of this configuration the throughput remains constant over time irrespective of the changing volume of data Adding more worker nodes than the shards could hardly be helpful in this case Introducing Auto Sharding for BigQueryIn order to eliminate the painful hand tuning of parallelism for the BigQuery sink we are introducing a new capability Auto Sharding to Dataflow Streaming Engine With Auto Sharding Dataflow dynamically adjusts the number of shards for BigQuery writes and balances the load to keep up with the input rate The following illustration shows how sharding changes over time in response to changing input data volume BenefitsTo determine the extent of the benefit we ran load tests to compare the performance of BigQuery I O connector with and without Auto Sharding To get a comprehensive understanding of the performance benefits we compared the throughput for worker counts ranging from to As you can see in the following charts Auto Sharding outperformed manual and fixed sharding with BigQuery streaming inserts It boosted the throughput by x to x in most cases without any other changes In general increase in throughput per worker has the potential to reduce the number of workers needed and thereby cost The difference in throughput increases as the number of workers is increased This is particularly important since it becomes very challenging to hand pick sharding configuration as the scale of your pipeline grows in order to handle increased input volume Streaming Auto Sharding also works with BigQuery batch loading of data and yields similar improvements to increase in throughput How to enable Auto Sharding for BigQuery SinksYou can enable Auto Sharding in Streaming Engine by setting a new option when configuring the BigQuery I O connector in your code The option is available in both streaming inserts and file loads modes starting Beam for Java and Beam for Python Alternatively you can opt in Auto Sharding for streaming inserts using Beam Java SDK with an additional Dataflow experiment experiments enable streaming auto sharding The following are code samples to enable Auto Sharding Java Python With the new option the sharding for the BigQuery writes will be automatically determined at runtime to make the best use of workers available If autoscaling is turned on sharding will be adjusted to keep all workers busy when the number of workers changes While the number of shards is not explicitly exposed the changes in sharding is reflected in job metrics such as data freshness and throughput For example increased system latency triggers more workers which in turn increases the number of shards As a result throughput increases and the backlog is cleared faster You can use the Dataflow Console to monitor the number of workers and the throughput of the BigQuery write stage To get started please refer to the documentation for the BigQuery IO Connector and the introduction page to learn more about different ways of data ingestion to BigQuery and how to choose one that fits your needs |
2021-05-25 08:00:00 |
GCP |
Cloud Blog JA |
Google Workspace の活用でコラボレーションを変革 |
https://cloud.google.com/blog/ja/products/workspace/next-evolution-of-collaboration-for-google-workspace/
|
本日より、チームですでに作業しているGoogleドキュメント、Googleスプレッドシート、スライドから直接、ウェブ上のGoogleMeetの通話にコンテンツを表示する機能を展開します。 |
2021-05-25 09:00:00 |
コメント
コメントを投稿