投稿時間:2020-10-22 05:30:19 RSSフィード2020-10-22 05:00 分まとめ(32件)

カテゴリー等 サイト名等 記事タイトル・トレンドワード等 リンクURL 頻出ワード・要約等/検索ボリューム 登録日
AWS AWS AWS 執行役員 技術統括本部長 岡嵜のご紹介 / Meet Tadashi Okazaki - Director, Head of Solutions Architect AWS Japan https://www.youtube.com/watch?v=USYo8h0n7mw AWS執行役員技術統括本部長岡嵜のご紹介MeetTadashiOkazakiDirectorHeadofSolutionsArchitectAWSJapanAWSJapanの執行役員技術統括本部長岡嵜をご紹介します。 2020-10-21 19:14:39
AWS AWS Enable Amazon VPC Access for AWS CodeBuild https://www.youtube.com/watch?v=wlsVsEwnBxA Enable Amazon VPC Access for AWS CodeBuildIn this video you ll see how to enable Amazon Virtual Private Cloud Amazon VPC access for AWS CodeBuild projects With this solution you can increase the security of your build process invoke CloudBuild privately and draw upon any resource hosted in your VPC for CodeBuild projects AWS CodeBuild is a fully managed build service in the cloud Ordinarily CodeBuild cannot access resources in a VPC However it s possible to enable access through your VPC specifications and CodeBuild project configuration Learn more about AWS CodeBuild at Subscribe More AWS videos More AWS events videos AWS AWSDemo 2020-10-21 19:02:35
AWS AWS - Webinar Channel AWS Outposts: Storage Foundations - AWS Online Tech Talks https://www.youtube.com/watch?v=A_khazmf6jU AWS Outposts Storage Foundations AWS Online Tech TalksS on Outposts makes it easy for you to store and retrieve data in your on premises Outposts environments using Amazon S APIs AWS Outposts is a fully managed service that extends AWS infrastructure services APIs and tools to virtually any datacenter co location space or on premises facility for a truly consistent hybrid experience S on Outposts is ideal for workloads that have data residency requirements to process and store data on premises or that need access to data for local processing In this tech talk you will learn the foundations of S on Outposts typical use cases and how to get started Learning Objectives Learn about Amazon S on Outposts foundations Discover use cases for S on Outposts Find out how to get started with S on Outposts To learn more about the services featured in this talk please visit 2020-10-21 19:14:22
python Pythonタグが付けられた新着投稿 - Qiita [初心者向け]Pythonで音声認識をしてLineに通知! https://qiita.com/taruscript/items/45a6a63568df202595c0 初心者向けPythonで音声認識をしてLineに通知はじめにGoogleのspeechtotextAPIとLineNotifyを最近使い始めたのですが、案外簡単に扱えると感じたので、今回はこのつの技術を使ったボットの作り方を説明します今回つくるもの状況が、上記画像のような状況で人が話した内容が文字列となって、それをLineNotifyを使ってLINEに出力するというものを開発します。 2020-10-22 04:50:28
python Pythonタグが付けられた新着投稿 - Qiita Django Serializerを使って関連先のフィールドを取得する https://qiita.com/AJIKING/items/a604e2d4e75cf1a1686b DjangoSerializerを使って関連先のフィールドを取得するSerializerを簡単にいうとモデルオブジェクトをJSONに変換してくれるそれでは今回はBookオブジェクトを取得する際に、ForeignKeyで参照しているAuthorをオブジェクトも含めて取得しますmodelを定義するサンプルのモデルですBookモデルはAuthorモデルを参照していますmodelspyclassAbstractModelmodelsModelidmodelsUUIDFieldprimarykeyTruedefaultuuiduuideditableFalseisdeletedmodelsCharFieldmaxlengthdefaultcreatedatmodelsDateTimeFieldautonowaddTrueupdatedatmodelsDateTimeFieldautonowTrueclassMetaabstractTrueclassAuthorAbstractModelfirstnamemodelsCharFieldmaxlengthlastnamemodelsCharFieldmaxlengthclassBookAbstractModeltitlemodelsCharFieldmaxlengthsubtitlemodelsCharFieldmaxlengthpricemodelsDecimalFieldmaxdigitsdecimalplacesblankTruenullTrueauthormodelsForeignKeyAuthorondeletemodelsPROTECTblankTruenullTrue参照元のモデルも一緒に取得するViewを定義するgenericsのListAPIViewを使って取得APIを作成しますauthorモデルも取得するのでquerysetにはselectrelatedでauthorを指定しておきます。 2020-10-22 04:07:41
Program [全てのタグ]の新着質問一覧|teratail(テラテイル) objdump の読み方を教えてください。 https://teratail.com/questions/299554?rss=all fffff 2020-10-22 04:01:36
Docker dockerタグが付けられた新着投稿 - Qiita Laradockでもfish shellを使いたい!! https://qiita.com/poposuke/items/43d6bd6d1e1a02322d8d Laradockでもfishshellを使いたいやることlaradockで特にファイルをいじらずにlaravelの環境構築をするとデフォルトでzshが選択されているのでfishに変更します。 2020-10-22 04:35:58
海外TECH Ars Technica Trial to deliberately infect people with coronavirus draws mixed reaction https://arstechnica.com/?p=1716302 speed 2020-10-21 19:55:19
Apple AppleInsider - Frontpage News Apple lands 'The Velvet Underground' documentary from director Todd Haynes https://appleinsider.com/articles/20/10/21/apple-lands-the-velvet-underground-documentary-from-director-todd-haynes Apple lands x The Velvet Underground x documentary from director Todd HaynesApple has landed the rights to The Velvet Underground an upcoming documentary that explores how the band changed the world of music set to air exclusively on Apple TV Image Credit AppleThe documentary serves to show how The Velvet Underground became a cultural touchstone ーone of rock n roll s most revered bands ーcapable of embodying the band s creative ethos how to be elegant and how to be brutal Read more 2020-10-21 19:09:53
Apple AppleInsider - Frontpage News Jabra adds active noise cancellation to Elite 75t, Elite Active 75t earbuds https://appleinsider.com/articles/20/10/21/jabra-adds-active-noise-cancellation-to-elite-75t-elite-active-75t-earbuds Jabra adds active noise cancellation to Elite t Elite Active t earbudsJabra has just released a firmware update for its Elite t and Elite Active t true wireless earbuds that introduces active noise cancellation bringing them more in line with Apple s AirPods Pro Jabra Elite t true wireless earbudsWe compared the Elite t as well as the Elite Active t to Apple s AirPods Pro They fared quite well with the biggest difference being the lack of ANC Read more 2020-10-21 19:28:30
海外TECH Engadget Quibi is reportedly shutting down https://www.engadget.com/quibi-shutting-down-report-195306262.html Quibi is reportedly shutting downQuibi the short form video streaming service that made a splash at CES with its Turnstyle feature is shutting down according to separate reports from The Wall Street Journal and The Information The Wall Street Journal reports Quibi founder Je 2020-10-21 19:53:06
海外TECH Engadget Boston Dynamics will start selling arms for its robodog Spot next year https://www.engadget.com/boston-dynamics-will-start-selling-arms-for-its-robodog-spot-next-year-192343407.html Boston Dynamics will start selling arms for its robodog Spot next yearBoston Dynamics has reportedly already sold more than of its Spot robots since starting commercial sales back in June Interested and deep pocketed parties can purchase one directly from the company s website as well as a host of accessor 2020-10-21 19:23:43
海外科学 NYT > Science Purdue Pharma Pleads Guilty to Criminal Charges for Opioid Sales https://www.nytimes.com/2020/10/21/health/purdue-opioids-criminal-charges.html Purdue Pharma Pleads Guilty to Criminal Charges for Opioid SalesThe Justice Department announced an billion settlement with the company Members of the Sackler family will pay million in civil penalties but criminal investigations continue 2020-10-21 19:15:45
海外科学 NYT > Science NASA’s OSIRIS-REX Mission Completes Quick Touch of Bennu Asteroid https://www.nytimes.com/2020/10/20/science/osiris-rex-mission.html NASA s OSIRIS REX Mission Completes Quick Touch of Bennu AsteroidThe spacecraft attempted to suck up rocks and dirt from the asteroid which could aid humanity s ability to divert one that might slam into Earth 2020-10-21 19:28:30
医療系 医療介護 CBnews コロナ禍で収益を改善させた病院の“秘策”とは?-患者とスタッフの安心感の確保こそ近道 https://www.cbnews.jp/news/entry/20201021095353 ldquo 2020-10-22 05:00:00
ニュース BBC News - Home Covid: South Yorkshire to move into tier 3 from Saturday https://www.bbc.co.uk/news/uk-54627017 covid 2020-10-21 19:19:58
ニュース BBC News - Home Covid: No safety concerns found with Oxford vaccine trial after Brazil death https://www.bbc.co.uk/news/world-latin-america-54634518 death 2020-10-21 19:25:38
ニュース BBC News - Home Angela Rayner apologises after "scum" remark in Commons https://www.bbc.co.uk/news/uk-politics-54638267 coronavirus 2020-10-21 19:37:47
ニュース BBC News - Home Purdue Pharma to plead guilty in $8bn opioid settlement https://www.bbc.co.uk/news/business-54636002 oxycontin 2020-10-21 19:16:40
ニュース BBC News - Home Covid-19: Nottingham party students fined £40,000 https://www.bbc.co.uk/news/uk-england-nottinghamshire-54631524 nottingham 2020-10-21 19:21:48
ビジネス ダイヤモンド・オンライン - 新着記事 「ハンコ警察」の大誤解、ムダな印鑑を一掃しても社会の効率は良くならない - 情報戦の裏側 https://diamond.jp/articles/-/251968 既得権益 2020-10-22 04:55:00
ビジネス ダイヤモンド・オンライン - 新着記事 125人の完全リモートワーク集団を率いるリーダーも実践「チーミング」とは - 新しいマネジメントの教科書 https://diamond.jp/articles/-/251332 取り組み 2020-10-22 04:50:00
ビジネス ダイヤモンド・オンライン - 新着記事 コロナ「1割減経済」での利益確保には、正規雇用者の削減が避けられない - 野口悠紀雄 新しい経済成長の経路を探る https://diamond.jp/articles/-/251967 正規雇用者 2020-10-22 04:45:00
ビジネス ダイヤモンド・オンライン - 新着記事 中国生まれの「歌舞伎町案内人」が、27年かけて日本に帰化した理由 - News&Analysis https://diamond.jp/articles/-/251881 newsampampanalysis 2020-10-22 04:40:00
ビジネス ダイヤモンド・オンライン - 新着記事 バイデン候補に強烈な追い風?民主社会主義が米国人を魅了するワケ - 内側から見た米国大統領選挙2020 https://diamond.jp/articles/-/251723 上院議員 2020-10-22 04:35:00
ビジネス ダイヤモンド・オンライン - 新着記事 人口の東京一極集中は本当に終わる?家選びで失敗しない通説の見極め方 - ビッグデータで解明!「物件選び」の新常識 https://diamond.jp/articles/-/251964 東京一極集中 2020-10-22 04:25:00
ビジネス ダイヤモンド・オンライン - 新着記事 誰にも話せない「心の毒」をデトックス!「オンライン坊主BAR」とは? - 消費インサイド https://diamond.jp/articles/-/251963 2020-10-22 04:20:00
ビジネス ダイヤモンド・オンライン - 新着記事 総合商社の業界研究!元伊藤忠人事が語る30年の変化と求められる人材像 - 親と子のための業界・企業研究 https://diamond.jp/articles/-/251797 企業研究 2020-10-22 04:15:00
ビジネス ダイヤモンド・オンライン - 新着記事 コロナ禍が変えた学校と教育、「大学爆破予告」急増の背景にあるものとは - 中学受験への道 https://diamond.jp/articles/-/251893 2020-10-22 04:10:00
ビジネス ダイヤモンド・オンライン - 新着記事 慢性痛の根本的な解決を目指し、集学的治療を究めてきた医師の信念 - 木原洋美「究める医師」の仕事と哲学 https://diamond.jp/articles/-/251930 慢性痛の根本的な解決を目指し、集学的治療を究めてきた医師の信念木原洋美「究める医師」の仕事と哲学名医やトップドクターと呼ばれる医師、ゴッドハンド神の手を持つといわれる医師、患者から厚い信頼を寄せられる医師、その道を究めようとする医師を取材し、仕事ぶりや仕事哲学などを伝える。 2020-10-22 04:05:00
ビジネス 東洋経済オンライン クボタ「自動田植え機」普及への期待とハードル 車だけじゃない!農機で進む自動運転の最前線 | 素材・機械・重電 | 東洋経済オンライン https://toyokeizai.net/articles/-/383304?utm_source=rss&utm_medium=http&utm_campaign=link_back 安全対策 2020-10-22 04:50:00
GCP Cloud Blog Meeting the need for speed with a cloud data warehouse https://cloud.google.com/blog/products/data-analytics/e-commerce-data-warehouse-migration/ Meeting the need for speed with a cloud data warehouseIn our work with Google Cloud customers we hear great stories of growth change and cloud success We worked closely with Trendyol Group a fast growing e commerce company based in Turkey on their data warehouse migration project Trendyol employs about people and the company s e commerce site gets about billion page views per year billion visits per year and million monthly unique users For this digital native company data has been at the heart of their business  Trendyol Group was facing unprecedented growth and the Trendyol data warehouse DWH team had been challenged with the performance and scalability of their existing Vertica data warehouse especially during the holiday shopping season and other busy retail seasons Performance issues had become critical over the past months and had business impact The DWH team realized that not being able to process data and deliver internal reports and dashboards on time was causing lost revenue and inaccurate supplier management For example the business couldn t react quickly when suppliers made bad decisions or sold a product that didn t actually have inventory   The capacity limitations of the on premises data warehouse forced the IT team to constantly tune performance and plan and scale capacity instead of focusing on business insights Trendyol s reporting team serves more than users with roughly workbooks and views in Tableau Prior to migration Trendyol stored over TB of data in their Vertica environment In addition there were over slowly changing dimensions SCDs in the ETL pipelines requiring the team to update of the data every day which led to an TB truncate insert during the ELT process  The size of the data was weighing down the business Business users couldn t meet SLAs for their Monday morning financial reports required by the executives To meet those busy periods their IT team had to spend time tuning workloads by killing long running queries to ensure timely completion of the reports For example business users couldn t run queries that spanned a three year period for aggregations due to capacity issues they could only do a one year time frame By the time that business users accessed the report the data was already stale Then when Thursdays came and users weren t running as many queries the DWH team found themselves with excess capacity  With the impact of COVID Trendyol needed to be able to react quickly and cut off non compliant products or suppliers to be able to meet the sudden increase in demand  The DWH team knew they needed to auto scale the workloads in a cost effective way They extended their Vertica environment for one more year while they started evaluating cloud data warehousing alternatives  Cloud data warehouse decision criteria The Trendyol team decided to look into a number of vendors including Snowflake and Google Cloud Their decision criteria for a cloud data warehouse included Instant scalability Given the variability in their analytical workload this was a critical need so they could have capacity on demand to run the Monday morning reports  Reduced operational costs Since the retail business is seasonal Trendyol needed to keep their costs low in line with demand  Uptime SLAs Their analytical platform needed to be highly available to meet business needs especially in these critical times BigQuery now offers a SLA Backup and recovery This is important so the team can look back in time in case there are errors in processing  Security This is a key requirement for them since they need to restrict access to personally identifiable information PII and sensitive data depending on roles  Ease of use It was very important that business users could transition to the new cloud data warehouse platform without a learning curve and could be productive immediately Evaluating cloud data warehousesBigQuery s comprehensive documentation and simple management interface let the Trendyol team set up BigQuery and fine tune queries for their proof of concept trial Other data warehouse vendors trials required a consultant to optimize and tune the environment They were able to move the data into BigQuery by themselves and it just worked They also used BigQuery features like time travel which met backup and recovery requirements out of the box and integrated Cloud Identity and Access Management Cloud IAM roles that met security requirements easily The most important feature in BigQuery for Trendyol was the separation of storage and compute so that they would not have to pay for compute when not in use Furthermore it was easy to scale their workload up and down without the need for startup or shutdown time which other tools required The DWH team ran through a comprehensive evaluation of alternative data warehouse tools with a variety of benchmarks to represent their main workloads including ELT end to end BI integration with BI and a number of different OLAP queries BigQuery was the preferred option for price and performance for each of the workloads  Here are three example queriesーOLAP type queries with joins involving a billion rows Regex analytical functions   Ad hoc queries representing power users Join four tables high cartesian joins m m m k Regex functions x dist count ELT publish layer with analytical functions Join five tables rows m m m m k analytical functions first last value group by Example of publish layer Join tables including subqueries rows m m m m x group byTesting results from TrendyolConcurrent queries BigQuery was the most cost effective and faster compared to the alternative solution BigQuery allowed testing increased slots and sharing resources seamlessly across reservations which wasn t possible in the alternative solution DML statements performance Similar across platforms including CTAS updates inserts but BigQuery was the most cost effective  End to end runtime With BI run BigQuery was faster Ingestion times BigQuery was an order of magnitude faster  Data ingestion benchmark Parquet files with size of GB million rows columns snappy compressed ELT s SCD phase With one of the largest dimensions creates more than million updates and approximately million inserts Overall BigQuery provided the best price for performance and its predictable flat rate pricing was key for the decision In the past the DWH team had purchased capacity ahead of time and often thought that it would end up being utilized but didn tーcreating significant cost and unpredictability The team would now be able to predict how much capacity they would use at the end of each month And the ability to scale up and down in minute intervals with Flex Slots was not available from any other vendor   Migrating to BigQueryThe Trendyol DWH team separated the migration into three main categories BI Tableau migration was done in two weeks The team changed source tables accessed by worksheets and weave reports Since Tableau has a native connection to BigQuery it was easy to migrate They used the same table and column names in BigQuery matching the ones Tableau reports are using and it just worked They also avoided using custom SQL in Tableau eliminating the need to rewrite most of the reports The team found BigQuery s ANSI SQL compliant dialect to be compatible with most of their requirements Additionally they had some custom SQL with a good amount of regular expressions which were easily addressed by writing around UDFs  ETL More than ETL jobs are scattered across three tools Attunity custom Python scripts and Kafka Connect The team has been doing ETL on prem and now in the second phase of the migration they ve started migrating ETL to BigQuery Data There was TB to start in Vertica that the team moved to BigQuery They used Attunity for SQL Server and Kafka Connect for cloud based sources In addition custom Python code integrated natively with the BigQuery JDBC driver Within three months the team ingested TB into BigQuery an order of magnitude larger than they had expected   Currently the Trendyol team stores TB of data and SCDs in BigQuery and processes TB of data daily They mix and match flat rate slot reservations and Flex Slots to get the best pricing at any given time For example they can now handle fluctuations in demand by purchasing Flex Slots on an on demand basis The data team can now concentrate on creating value rather than spending time on operationalizing the data warehouse  The relationship between the IT and business teams has been transformed too There are now plenty of compliments on speed and scalability The data team can now produce reports in an hour on Monday morning meeting their SLAs comfortably The ODS pipeline previously took two to three hours depending on the day Trendyol s BigQuery migration has helped restore trust between the IT and business teams enable data driven decision making save on costs and meet customer needs quickly  Learn more about Trendyol and about BigQuery s data warehouse migration program 2020-10-21 20:00:00

コメント

このブログの人気の投稿

投稿時間:2021-06-17 05:05:34 RSSフィード2021-06-17 05:00 分まとめ(1274件)

投稿時間:2021-06-20 02:06:12 RSSフィード2021-06-20 02:00 分まとめ(3871件)

投稿時間:2020-12-01 09:41:49 RSSフィード2020-12-01 09:00 分まとめ(69件)