投稿時間:2022-03-28 05:17:45 RSSフィード2022-03-28 05:00 分まとめ(22件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
海外TECH	MakeUseOf	7 Reasons Why You Should (and Shouldn’t) Dual Boot Linux	https://www.makeuseof.com/tag/reasons-dual-boot-linux/	Reasons Why You Should and Shouldn t Dual Boot LinuxOften making the choice of installing Linux over Windows can be tricky to justify But why not opt for both Here are five reasons to dual boot and two reasons you shouldn t	2022-03-27 19:30:14
海外TECH	MakeUseOf	Why Does Linux Have So Many Distributions? Linux Distros Explained	https://www.makeuseof.com/why-so-many-linux-distributions/	Why Does Linux Have So Many Distributions Linux Distros ExplainedThere are hundreds of active Linux distros currently and many more under development Why are there so many distributions and do we need all of them	2022-03-27 19:30:14
海外TECH	DEV Community	Big Data Processing, EMR with Spark and Hadoop | Python, PySpark	https://dev.to/wardaliaqat01/big-data-processing-emr-with-spark-and-hadoop-python-pyspark-4jo4	Big Data Processing EMR with Spark and Hadoop Python PySpark Introduction AWS s cool data analysis services can be of significant help when it comes to processing and analyzing large amounts of data Use Case To demonstrate our data processing job we will use EMR cluster and S as a storage medium for data along with Python code and the PySpark library We will execute python code on a data set of Stack Overflow Annual Developer Survey and print out some results based on that data Those results will then be stored in S In case you are just starting with Big Data I would like to introduce you to some terms we are going to work with You may skip below few lines if you re already familiar EMR Elastic MapReduce Amazon EMR is a managed cluster platform that simplifies running big data frameworks such as Apache Hadoop and Apache Spark on AWS to process and analyze vast amounts of data Wanna dig more dipper Amazon S Amazon S is object storage built to store and retrieve any amount of data from anywhere S is a global service Simply say it works like Google drive Wanna dig more dipper Apache Spark Apache Spark is an open source distributed processing system used for big data workloads Wanna dig more dipper Apache Hadoop Apache Hadoop is an open source framework that is used to efficiently store and process large datasets ranging in size from gigabytes to petabytes of data Wanna dig more dipper Hadoop is designed to handle batch processing efficiently whereas Spark is designed to handle real time data efficiently Hadoop is a high latency computing framework which does not have an interactive mode whereas Spark is a low latency computing and can process data interactively PySpark PySpark is the Python API for Apache Spark an open source distributed computing framework and set of libraries for real time large scale data processing Wanna dig more dipper OutlineDownloading a Data Set from Stack OverflowSetup an Amazon S bucket with different foldersSetting up an EMR clusterWrite a python code to perform some analysis on data and print results Connecting to Cluster via SSH and using PySpark to load data from Amazon SViewing analysis resultsCleanupRecap Guided Procedure Downloading a Data Set from Stack OverflowGo to Stack Overflow Annual Developer Survey and download the latest data set for Ultimately it will download four files but in this case we will only be using the Survey Results Public file Setup an Amazon S bucket with different foldersLogin to AWS Management ConsoleNavigate to S and bucketsCreate a new bucket named as big data demo bucket with Versioning enabled and encryption trueClick on your bucket once you ve created itCreate two folders named as bigdata emr logs for storing EMR logs and data source for storing our source data file with encryption enabledPlace your source data file in the source data folder Setting up an EMR clusterSearch for EMRClick on create clusterAn S bucket is needed to store EMR logs When you don t want to do it manually EMR will automatically create a bucket for you to store logs Select a bucket and folder that we created in previous stepSelect Spark with Hadoop and Zeppelin in the software configurationsIn terms of hardware configurations you can choose EC type based on your needsFor testing purposes instances would be sufficient You may create as many as you needEnable auto termination which will terminate your cluster if any error occurs during the creation processYou can also set the cluster to automatically terminate when it goes into idle state for longIt is also completely up to you whether or not to scaleSelect a key pair under security and access to SSH after launching the clusterIf you do not have a keypair you can create easily from EC dashboardOnce you click on the Create Cluster button your cluster will be created It usually takes to minutes for the cluster to become operational Write a python code to perform some analysis on data and print resultsNow let s write some code A spark job will run this code to analyze the data and print out the results from pyspark sql import SparkSessionfrom pyspark sql functions import colS DATA SOURCE PATH s big data demo bucket data source survey results public csv S DATA OUTPUT s big data demo bucket data output def main spark SparkSession builder appName BigDataDemoApp getOrCreate all data spark read csv S DATA SOURCE PATH header True print Total number of records in dataset s all data count selected data all data where col Country United States of America amp col AgestCode years print Total number of engineers who work more than hours in the US is s selected data count selected data write mode overwrite parquet S DATA OUTPUT print Selected data was successfully saved to s S DATA OUTPUT if name main main What does this code do Setting up a Spark sessionData reading from SPrinting some results based on certain conditionsA S folder is created for storing the results Connecting to Cluster via SSH and using PySpark to load data from Amazon SDon t forget to enable SSH connections before trying to SSH into your cluster The port needs to be added to the cluster s master security groupFollow the instructions accordingly to connect to your cluster if you are using Windows or MacI m on windows so I used putty for the connectionUse the vi main py command to create a Python file using the Vim editorPress I on your keyboard and paste your codePress the ECS key to exit the insert modeType wq to quite the editor with saving your changesIf you type cat main py you can view your codeTo submit this spark job use spark submit filename Your job will begin executing Viewing analysis resultsYour three print results can be viewed in the logs after completionYou can now see that the latest logs are stored in the logs folder in S Additionally you ll notice a new folder named data output that contains all output results with success files has been created for you CleanupYou can then terminate the cluster to save moneyWhen you terminate a cluster all EC associated with it will also be terminated RecapThis article showed how you can use EMR and Amazon S to process and analyze a vast amount of data collected from Stack Overflow developer survey to extract some useful insightsWelcome to the end of this article Happy Clouding Let me know what do you think about it	2022-03-27 19:03:00
海外TECH	DEV Community	Getting Started Machine Learning with AWS	https://dev.to/saimajavaid/getting-started-machine-learning-with-aws-20ai	Getting Started Machine Learning with AWS What is Machine Learning Machine learning ML is a software development technique that enable computers to solve complex problems by using samples from real world data Therefore this technique lets computer to automatically learn and improve with the experience when not being explicitly programmed to do so How it works In traditional problem solving with software engineering engineer will gather data and analyze the real world problem They design a solution and write down a code to solve that particular problem It s a laborious task to develop a code for many real world problems In Machine Learning developer writes a code snippet that will be trained by the real world data in such a way that it will start predicting the outcome of new data which was not supplied before That code snippet is called Model and it will be trained by a special program called Model Training Algorithm Why Machine Learning with AWS Yes that s the first question arises when you ll start reading the title Why AWS AWS is offering so many easy to use services and tools which makes it easy for everyone to dive in the machine learning journey It abstracted out so many complex implementation from the user and provides easy functionality that any new comer can easily start playing with The mission of AWS is to put machine learning in the hands of every developer AWS provides many pre trained AI services which can be used as a ready made intelligence to a wide range of applications There are many example to such services for example personalized recommendations modernizing your contact center improving safety and security and increasing customer engagement Amazon SageMakerIt s a fully managed service which removes complexity from machine learning workflows therefore every developer and data scientist can deploy machine learning easily Getting StartedAs I have previously explained that AWS is offering many services that enable every developer to easily start working with ML However AWS is also offering many ways of learning for the ML students and developers that are new to this field AWS has created an educational portfolio of devices that helps to put new machine learning techniques into the hands of developers in a fun ways with AWS DeepLens AWS DeepRacer and AWS DeepComposer AWS DeepLens This is a camera device that helps in learning Computer vision AWS DeepRacer It s a racing car designed to run on the physical track It helps to learn Reinforcement Learning technique AWS DeepComposer It s a composing device that powered by generative AI It creates melody that transforms into a completely original song AWS DeepLensAWS DeepLens allows you to create and deploy end to end computer vision based applications This device is integrated with several AWS machine learning services You can connect with AWS cloud to use deployed models that are trained and test with the new data captured by the device It enables you to learn and explore the latest artificial intelligence AI tools and techniques for developing computer vision applications based on a deep learning model AWS DeepRacerThis device is an autonomous racing car designed to test reinforcement learning models by racing on a physical track Reinforcement learning is a ML technique where a model normally called as agent here is trained by feedback from the environment The agent will be rewarded in numbers for the actions it takes Actions that help the agent achieve its goal are incentivized with higher numbers The actions that are not helpful will result in a low reward or no reward There are several racing competitions held monthly by AWS DeepRacer League You can enter in these competitions by sending your model for chances to win prizes while diving into machine learning development skills Note that the device is not mandatory during development AWS offers simulator where you can test your model There competitions also not required physical device many competitions are held using simulators AWS DeepComposerAWS DeepComposer is specifically design to provide a creative and easy way to get started with machine learning ML specifically generative AI It consists of a USB keyboard that connects with the computer to input melody There is an AWS DeepComposer console which includes a Music Studio to generate music and there is a learning capsules to dive deep into generative AI models Here again you do not need to purchase a device to sharpen your skill you can upload existing music and apply ML techniques There are several AWS DeepComposer Chartbusters challenges where you can share your music generated by ML techniques and showcase your skills with other AWS AccountYou will need an account to access AWS console and use ML services It requires payment information but that is only for verification and it won t deduct amount until you are using free tier and services that are free AWS offers a free tier that allows you to use several services free of cost up to specified limits for each service Details can be find on the AWS website Reference amp Further ReadingsAWS DeepComposer amazon com AWS DeepRacer the fastest way to get rolling with machine learning amazon com AWS DeepLens Deep learning enabled video camera for developers AWS amazon com	2022-03-27 19:01:40
海外TECH	Engadget	Apple and Google close loophole that allowed Russians to use Mir cards for mobile payments	https://www.engadget.com/apple-pay-google-pay-mir-loophole-closed-191434174.html?src=rss	Apple and Google close loophole that allowed Russians to use Mir cards for mobile paymentsApple has closed a loophole that had allowed some Russians to continue using its mobile payments service despite the ongoing economic sanctions against Russia According to Reuters the company told the country s largest lender on Thursday it would no longer support Russia s homegrown Mir payments system through Apple Pay quot Apple has informed NSPK it is suspending support for Mir cards in the Apple Pay payment service quot the National Card Payment System said Friday quot Starting from March th users cannot add new Mir cards to the service Apple will stop all operations of previously added cards over the next few days quot Google took similar action last week as well According to a separate report from The Wall Street Journal the company paused a pilot that had allowed Russians to connect their Mir cards to Google Pay quot Google Pay is pausing payments related services in Russia as a result of payment services disruption out of our control quot a Google spokesperson told the outlet As The Verge notes the Central Bank of Russia established Mir after the US and other countries imposed sanctions on Russia in response to its annexation of Crimea in According to statistics shared by the Central Bank Mir cards are involved in more than percent of all card transactions within the country Previously cards from major Russian financial institutions like VTB Group and Sovcombank stopped working with Apple Pay and Google Pay shortly after the Kremlin launched its invasion of Ukraine on February th	2022-03-27 19:14:34
医療系	医療介護 CBnews	コロナ直撃の20年度診療実態に迫る-先が見えない時代の戦略的病院経営（167）	https://www.cbnews.jp/news/entry/20220325192255	中央社会保険医療協議会	2022-03-28 05:00:00
ニュース	BBC News - Home	Ferry firms to meet officials over P&O fallout	https://www.bbc.co.uk/news/business-60895833?at_medium=RSS&at_campaign=KARANGA	falloutbosses	2022-03-27 19:32:30
ニュース	BBC News - Home	Afghanistan: Taliban bars BBC TV programmes from schedules	https://www.bbc.co.uk/news/world-asia-60893054?at_medium=RSS&at_campaign=KARANGA	content	2022-03-27 19:53:37
ビジネス	ダイヤモンド・オンライン - 新着記事	ソニー、ゲームとカメラが減収でも2桁増収まで押し上げた「ヒーロー」とは？ - ダイヤモンド 決算報	https://diamond.jp/articles/-/300180	ソニー、ゲームとカメラが減収でも桁増収まで押し上げた「ヒーロー」とはダイヤモンド決算報コロナ禍が年目に突入し、多くの業界や企業のビジネスをいまだに揺さぶり続けている。	2022-03-28 04:55:00
ビジネス	ダイヤモンド・オンライン - 新着記事	転職での退職代行サービス利用は絶対NG！「辞め太り」する人の賢いやり方 - 転職で幸せになる人、不幸になる人　丸山貴宏	https://diamond.jp/articles/-/300153	退職代行サービス	2022-03-28 04:50:00
ビジネス	ダイヤモンド・オンライン - 新着記事	おひとり様ならではのリスク管理、「死亡保険金」より大事なお金がある！ - おひとり様がお金に困らない方法	https://diamond.jp/articles/-/299945	損害保険	2022-03-28 04:40:00
ビジネス	ダイヤモンド・オンライン - 新着記事	中国「ゼロコロナ政策」はいつまで続くか、政府が掲げる“経済5.5％成長”の足枷に - 政策・マーケットラボ	https://diamond.jp/articles/-/300081	中国「ゼロコロナ政策」はいつまで続くか、政府が掲げる“経済成長の足枷に政策・マーケットラボ中国のゼロコロナ政策は、新型コロナウイルスの感染が収束するまで継続するとみられる。	2022-03-28 04:35:00
ビジネス	ダイヤモンド・オンライン - 新着記事	失われた30年をリープ・フロッグ！感性を武器にした「新たな日本型組織モデル」 - フィジカルとテクノロジー	https://diamond.jp/articles/-/300104	前野隆司	2022-03-28 04:30:00
ビジネス	ダイヤモンド・オンライン - 新着記事	年収が高い会社ランキング2021【従業員の平均年齢30代前半】1位と2位はM＆A仲介！ - ニッポンなんでもランキング！	https://diamond.jp/articles/-/300111	上場企業	2022-03-28 04:25:00
ビジネス	ダイヤモンド・オンライン - 新着記事	年収が高い会社ランキング2021【従業員の平均年齢30代前半・完全版】 - ニッポンなんでもランキング！	https://diamond.jp/articles/-/300013	上場企業	2022-03-28 04:25:00
ビジネス	ダイヤモンド・オンライン - 新着記事	「小室圭さんバッシング」が世間で止まらない3つの理由 - ＤＯＬ特別レポート	https://diamond.jp/articles/-/299947	眞子さま	2022-03-28 04:20:00
ビジネス	ダイヤモンド・オンライン - 新着記事	「暴落時こそ仕込み時！」ピーター・リンチ氏の格言に学ぶ、バーゲン価格で“10倍株の卵”を買う方法 - 今週の週刊ダイヤモンド　ここが見どころ	https://diamond.jp/articles/-/300006	「暴落時こそ仕込み時」ピーター・リンチ氏の格言に学ぶ、バーゲン価格で“倍株の卵を買う方法今週の週刊ダイヤモンドここが見どころ『週刊ダイヤモンド』月日号の第一特集は「暴落時こそ仕込む日米強い株」です。	2022-03-28 04:15:00
ビジネス	ダイヤモンド・オンライン - 新着記事	倒産危険度ランキング【外食25社】「禁酒令」で大打撃の居酒屋が上位独占 ［2021年度ベスト4］ - DOLベスト記事アワード	https://diamond.jp/articles/-/300161	関連	2022-03-28 04:10:00
ビジネス	ダイヤモンド・オンライン - 新着記事	山陽新幹線が開業50周年、航空機とのシェア争いで果たした役割とは - News&Analysis	https://diamond.jp/articles/-/300167	newsampampanalysis	2022-03-28 04:05:00
北海道	北海道新聞	空港産ホップでビール造り　南紀白浜、活性化に期待	https://www.hokkaido-np.co.jp/article/661916/	南紀白浜	2022-03-28 04:31:00
北海道	北海道新聞	陸自に「電子作戦隊」発足　安保新領域、大幅強化へ	https://www.hokkaido-np.co.jp/article/661915/	陸上自衛隊	2022-03-28 04:31:00
ビジネス	東洋経済オンライン	大阪新駅､工事中に｢駅名標｣設置した納得の理由 ｢イノベーションの実験場｣として2023年春開業 | 駅･再開発 | 東洋経済オンライン	https://toyokeizai.net/articles/-/541547?utm_source=rss&utm_medium=http&utm_campaign=link_back	特急列車	2022-03-28 04:30:00