投稿時間:2022-06-27 15:16:25 RSSフィード2022-06-27 15:00 分まとめ(22件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
IT	ITmedia 総合記事一覧	[ITmedia ビジネスオンライン] 「NOT A HOTEL」を1日単位で利用できるNFT販売　権利の期間は47年間	https://www.itmedia.co.jp/business/articles/2206/27/news101.html	itmedia	2022-06-27 14:53:00
IT	ITmedia 総合記事一覧	[ITmedia News] 最近話題の「USB」って何の略？	https://www.itmedia.co.jp/news/articles/2206/27/news133.html	itmedia	2022-06-27 14:30:00
IT	ITmedia 総合記事一覧	[ITmedia ビジネスオンライン] 「屋外ではマスクをはずして」　厚生労働省、連日SNSで注意喚起	https://www.itmedia.co.jp/business/articles/2206/27/news128.html	注意喚起	2022-06-27 14:26:00
IT	ITmedia 総合記事一覧	[ITmedia PC USER] サードウェーブ、14型モバイルノート「F-14TG」にCore i7搭載の上位モデル	https://www.itmedia.co.jp/pcuser/articles/2206/27/news131.html	corei	2022-06-27 14:11:00
python	Pythonタグが付けられた新着投稿 - Qiita	プログラミング未経験者が育休中にKagglerになるまでの記録	https://qiita.com/data_mom/items/164c11501556b77a1b5a	kaggl	2022-06-27 14:54:37
js	JavaScriptタグが付けられた新着投稿 - Qiita	[JavaScript]filter()で連想配列を指定条件で絞り込む	https://qiita.com/HuntingRathalos/items/e0f2915a2497d3d02f0b	filter	2022-06-27 14:15:06
AWS	AWSタグが付けられた新着投稿 - Qiita	AWS Certified Database - Specialty(DBS-C01) を取得しました	https://qiita.com/ttuser/items/d83e4448ca0c5fedf626	ieddatabasespecialtydbsc	2022-06-27 14:43:28
Git	Gitタグが付けられた新着投稿 - Qiita	git diff をgithubみたいに見やすくしたい人へ　(左右画面分割diff , 差分ハイライト	https://qiita.com/momoka0122y/items/9e5ca351791485aa85ff	gitdiff	2022-06-27 14:07:51
Ruby	Railsタグが付けられた新着投稿 - Qiita	FactoryBotの存在理由（自分用）	https://qiita.com/aono1234/items/b0bc3dd42ef67dd14e63	factorybot	2022-06-27 14:23:00
海外TECH	DEV Community	Build a Better GitHub Insight Tool in a Week? A True Story	https://dev.to/ossinsight/build-a-better-github-insight-tool-in-a-week-a-true-story-14en	Build a Better GitHub Insight Tool in a Week A True StoryIn early January Max our CEO a big fan of open source asked if my team could build a small tool to help us understand all the open source projects on GitHub and that if everything worked well we should open the API to help open source developers to build better insights In fact GitHub continuously publishes the public events in its open source world through the open API Thank you and well done Github We can certainly learn a lot from the data I was excited about this project until Max said “You ve only got one week Well the boss is the boss Although time was tight and we were faced with multiple head aching problems I decided to take up this challenge Headache we need both historical and real time data After some quick research we found GHArchive an open source project that collects and archives all GitHub data from and updates it hourly By the way a lot of open source analytical tools such as CNCF s Devstats rely on GH Archive too Thanks to GH Archive we found the data source But there s another problem hourly data is good but not good enough We wanted our data to be updated in real timeーor at least near real time We decided to directly use the GitHub event API which collects all events that have occurred within the past hour By combining the data from the GH Archive and the GitHub event API we can gain streaming real time event updates Headache the data is huge After we decompressed all the data from GH Archive we found there were more than billion rows of GitHub events That s a lot of data We also noticed that about rows were generated and updated each hour The database solution would be tricky here Our goal is to build an application that provides real time data insights based on a continuously growing dataset So scalability is a must NoSQL databases can provide good scalability but what follows is how to handle complex analytical queries Unfortunately NoSQL databases are not good at that Another option is to use an OLAP database such as ClickHouse ClickHouse can handle the analytical workload very well but it is not designed for serving online traffic If we chose it we would need another database for the online traffic What about sharding the database and then building an extract transform load ETL pipeline to synchronize the new events to a data warehouse This sounds workable According to our product manager s PM s plan we needed to do some repo specific or user specific analysis Although the total data volume was huge the number of events was not too large for a single project or user This meant using the secondary indexes in RDBMS would be a good idea But if we decided to use the above architecture we had to be careful in selecting the database sharding key For example if we use user id as the sharding key then queries based on repo id will be very tricky Another requirement from the PM was that our insight tool should provide OpenAPI which meant we would have unpredictable concurrent traffic from the outside world Since we re not experts on Kafka and data warehouses mastering and building such an infrastructure in just one week was a very difficult task for us The choice is obvious now and don t forget PingCAP is a database company TiDB seems a perfect fit for this and it s a good chance to eat our own dog food So why not using TiDB If we use TiDB can we get SQL support including complex amp flexible queries ️Scalability ️Secondary index support for fast lookup ️Capability for online serving ️Wow It seems we got a winner To choose a database to support an application like OSS Insight we think TiDB is a great choice Plus its simplified technology stack means a faster go to market and faster delivery of my boss assignment After we used TiDB we got a simplified architecture as shown below Headache We have a pushy PM Just as the subtitle indicates we have a very “pushy PM which is not always a bad thing His demands kept extending from the single project analysis at the very beginning to the comparison and ranking of multiple repositories and to other multidimensional analysis such as the geographical distribution of stargazers and contributors What s more pressing was that the deadlines stayed unchanged We had to keep a balance between the growing demands and the tight deadlines To save time we built our website using Docusaurus an open source static site generator in React with scalability rather than building a site from scratch We also used Apache Echarts a powerful charting library to turn analytical results into good looking and easy to understand charts We chose TiDB as the database to support our website and it perfectly supports SQL This way our back end engineers could write SQL commands to handle complex and flexible analytical queries with ease and efficiency Then our front end engineers would just need to display those SQL execution results in the form of good looking charts Finally we made it We prototyped our tool in just one week and named it OSS Insight short for open source software insights We continued to fine tune it and it was officially released on May How we deal with analytical queries with SQLLet s use one example to show you how we deal with complex analytical queries Analyze a GitHub collection JavaScript frameworksOSS Insight can analyze popular GitHub collections by many metrics including the number of stars issues and contributors Let s identify which JavaScript framework has the most issue creators This is an analytical query that includes aggregation and ranking To get the result we only need to execute one SQL statement In the statement above the collections and collection items tables store the data of all GitHub repository collections in various areas Each table has rows To get the order of issue creators we need to associate the repository ID in the collection items table with the real billion row github events table as shown below Next let s look at the execution plan TiDB is compatible with MySQL syntax so its execution plan looks very similar to that of MySQL In the figure below notice the parts in red boxes The data in the table collection items is read through distributed row which means this data is processed by TiDB s row storage engine TiKV The data in the table github events is read through distributed column which means this data is processed by TiDB s columnar storage engine TiFlash TiDB uses both row and columnar storage engines to execute the same SQL statement This is so convenient for OSS Insight because it doesn t have to split the query into two statements TiDB returns the following result Then we just need to draw the result with Apache Echarts into a more visualized chart as shown below Note You can click the REQUEST INFO on the upper right side of each chart to get the SQL command for each result Feedback People love it After we released OSS Insight on May we have received loud applause on social media via emails and private messages from many developers engineers researchers and people who are passionate about the open source community in various companies and industries I am more than excited and grateful that so many people find OSS Insight interesting helpful and valuable I am also proud that my team made such a wonderful product in such a short time Lessons learnedLooking back at the process we used to build this website we have learned many mind refreshing lessons First quick doesn t mean dirty as long as we make the right choices Building an insight tool in just one week is tricky but thanks to those wonderful ready made and open source projects such as TiDB Docusaurus and Echarts we made it happen with efficiency and without compromising the quality Second it s crucial to select the right databaseーespecially one that supports SQL TiDB is a distributed SQL database with great scalability that can handle both transactional and real time analytical workloads With its help we can process billions of rows of data with ease and use SQL commands to execute complicated real time queries Further using TiDB means we can leverage its resources to go to market faster and get feedback promptly If you like our project or are interested in joining us you re welcome to submit your PRs to our GitHub repository You can also follow us on Twitter for the latest information More content and specific SQL can go into the official website to learn more	2022-06-27 05:31:01
ニュース	BBC News - Home	NI Protocol: MPs to vote on plans to ditch parts of Brexit deal	https://www.bbc.co.uk/news/uk-northern-ireland-61946333?at_medium=RSS&at_campaign=KARANGA	international	2022-06-27 05:31:51
ニュース	BBC News - Home	Sturgeon summit brings abortion protest law closer	https://www.bbc.co.uk/news/uk-scotland-61918888?at_medium=RSS&at_campaign=KARANGA	buffer	2022-06-27 05:31:50
ニュース	BBC News - Home	Japan urges 37 million people to switch off lights	https://www.bbc.co.uk/news/business-61947315?at_medium=RSS&at_campaign=KARANGA	tokyo	2022-06-27 05:34:36
ニュース	BBC News - Home	Manchester United: The pre-season challenges that face new boss Erik ten Hag	https://www.bbc.co.uk/sport/football/61941600?at_medium=RSS&at_campaign=KARANGA	Manchester United The pre season challenges that face new boss Erik ten HagPre season looms for Manchester United and new boss Erik ten Hag will finally begin his journey as the club s fifth permanent manager since Sir Alex Ferguson retired in	2022-06-27 05:25:34
ビジネス	ダイヤモンド・オンライン - 新着記事	香港、揺らぐ国際金融ハブとしての地位 - WSJ発	https://diamond.jp/articles/-/305547	金融ハブ	2022-06-27 14:20:00
北海道	北海道新聞	大リーグ、エンゼルス３位に後退　パドレスは２位で変わらず	https://www.hokkaido-np.co.jp/article/698517/	大リーグ	2022-06-27 14:51:00
北海道	北海道新聞	根室市と浜中町で震度２	https://www.hokkaido-np.co.jp/article/698514/	釧路管内	2022-06-27 14:49:50
北海道	北海道新聞	夏の高校野球支部予選・６月２７日の試合結果	https://www.hokkaido-np.co.jp/article/698423/	夏の高校野球	2022-06-27 14:39:13
北海道	北海道新聞	「仮面女子」猪狩さん敗訴　看板下敷き、国への請求棄却	https://www.hokkaido-np.co.jp/article/698509/	仮面女子	2022-06-27 14:18:06
IT	週刊アスキー	エスニック料理をビュッフェスタイルで楽しもう！　そごう横浜「SOGO YOKOHAMA BEER GARDEN」を開催	https://weekly.ascii.jp/elem/000/004/095/4095894/	sogoyokohamabeergarden	2022-06-27 14:50:00
IT	週刊アスキー	周辺機器に「サイバースティック」が登場！「メガドライブミニ2」の収録タイトル第2弾を公開	https://weekly.ascii.jp/elem/000/004/095/4095914/	収録予定	2022-06-27 14:40:00
IT	週刊アスキー	IIJ、フルMVNOサービス「IIJモバイルサービス/タイプI」でSIMカード納品後にプランやネットワークタイプを選べる「ストック品目」を提供開始	https://weekly.ascii.jp/elem/000/004/095/4095885/	提供開始	2022-06-27 14:10:00