投稿時間:2023-06-03 07:12:53 RSSフィード2023-06-03 07:00 分まとめ(16件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
AWS	AWS	Measure and Monitor Data Quality of your Datasets in AWS Glue Data Catalog | Amazon Web Services	https://www.youtube.com/watch?v=rK1NGQu4Sgg	Measure and Monitor Data Quality of your Datasets in AWS Glue Data Catalog Amazon Web ServicesAWS Glue Data Quality is a feature of AWS Glue that measures and monitors the data quality of data repositories In this video we provide an overview of what AWS Glue Data Quality is key capabilities and walk you through how you could use it to manage data quality for your data assets cataloged in AWS Glue Data Catalog You will gain a clear understanding of how to use this capability after you review this video Learn more about AWS Glue Data Quality at Subscribe More AWS videos More AWS events videos Do you have technical AWS questions Ask the community of experts on AWS re Post ABOUT AWSAmazon Web Services AWS is the world s most comprehensive and broadly adopted cloud platform offering over fully featured services from data centers globally Millions of customers ーincluding the fastest growing startups largest enterprises and leading government agencies ーare using AWS to lower costs become more agile and innovate faster AWS AmazonWebServices CloudComputing DataQuality DataGovernance AWSGlue AWSGlueDataQuality ETL DataCatalog GlueStudio DataIntegration	2023-06-02 21:21:03
Ruby	Rubyタグが付けられた新着投稿 - Qiita	validates :number, numericality: {in: 3..9} なんて使えないぞ？	https://qiita.com/scivola/items/515ed2cb6dab2b21a5a3	activerecord	2023-06-03 06:38:41
Ruby	Railsタグが付けられた新着投稿 - Qiita	validates :number, numericality: {in: 3..9} なんて使えないぞ？	https://qiita.com/scivola/items/515ed2cb6dab2b21a5a3	activerecord	2023-06-03 06:38:41
海外TECH	MakeUseOf	The 6 Best Live streaming Platforms to Watch	https://www.makeuseof.com/best-live-streaming-platforms-to-watch/	content	2023-06-02 21:45:18
海外TECH	MakeUseOf	How to Create a Custom Notification Component in React	https://www.makeuseof.com/create-custom-notification-component-in-react/	component	2023-06-02 21:30:18
海外TECH	MakeUseOf	How to Use Popular Social Media Platforms to Boost Your Well-Being	https://www.makeuseof.com/use-popular-social-media-platforms-boost-well-being/	How to Use Popular Social Media Platforms to Boost Your Well BeingTurn your relationship with social media from toxic to inspiring and productive with these tips on using the most popular platforms for good	2023-06-02 21:16:17
海外TECH	DEV Community	LB Labs: The Magic of Compound Efficiencies in Engineering	https://dev.to/linearb/lb-labs-the-magic-of-compound-efficiencies-in-engineering-2l24	LB Labs The Magic of Compound Efficiencies in EngineeringAs the milestone book Atomic Habits laid out the key to life changing habits is adopting one effectively and then layering another desirable habit on top of it The same is true for efficiencies in software engineering When your team adopts one efficiency sees it bear fruit then adds the next efficiency habit on top of it the result is compounding efficiencies In this conversation LinearB s CTO Yishai Beeri reveals the data on compound efficiencies as experienced by real dev teams out in the wild “There are dev orgs that just visit their metrics dashboard every three weeks That s nice but that s not enough That s not going to change behaviors that improve efficiency Episode Highlights Sourcing the data Visibility for devs amp managers Improving code reviews What are compounding efficiencies Streamlining the PR process Results from efficiencies gained Giving devs back more focus time How to get compounding efficiencies for your teamEpisode Transcription While you re here check out this video from our YouTube channel and be sure to like and subscribe when you do A part Summer Workshop Series for Engineering ExecutivesEngineering executives register now for LinearB s part workshop series designed to improve your team s business outcomes Learn the three essential steps used by elite software engineering organizations to decrease cycle time by on average and deliver better results Benchmark Automate and Improve Don t miss this opportunity to take your team to the next level save your seat today	2023-06-02 21:50:36
海外TECH	DEV Community	How to build your own data platform. Episode 2: authorization layer. Data Lake implementation.	https://dev.to/adevintaspain/how-to-build-your-own-data-platform-episode-2-authorization-layer-data-lake-implementation-22l9	How to build your own data platform Episode authorization layer Data Lake implementation Introduction This is the second episode in the series about how to build your own data platform You can find the whole list of articles in the following link gu martinm list how to build your own data platform efeceRemember a data platform will be used by many teams and users Also the data to be stored could be coming from many and different sources Data owners will want to set permissions and boundaries about who can access the data that they are storing in the data platform In this episode I will explain how you can add these capabilities to your data platform Also I will introduce the concept of data mesh and how you can use the authorization layer for implementing the workflow between data consumers and data owners that you will need for creating a successful data mesh Authorization layer Our authorization layer will be on the top of the storage one In this way users and applications willing to use the stored data will need to do it through this layer in a safe way No data will escape from the storage layer without authorization For implementing this layer you can use different solutions like Unity Catalog from Databricks Lake Formation from AWS plain IAM roles also from AWS Apache Ranger Privacera and many others For this article and because we are working with Amazon Web Services we will be implementing this layer using IAM roles and Lake Formation Processing layer Human users and processes will be the ones accessing the stored data through the authorization layer Machines and processes like Zeppelin notebooks AWS Athena for SQL clusters of AWS EMR Databricks etc etc The problem with the authorization Data engineers data analysts and data scientists work in different and sometimes isolated teams They do not want their data to be deleted or changed by tools or people outside their teams Also for being GDPR compliant to access PII data big restrictions will be required even at column or row level Every stored data needs to have an owner and in Data Mesh data owners are typically in charge of granting access to their data What is a Data mesh Taken from what is data meshThe term data mesh coined in by Zhamak Dehghani is based on four key principles Domain ownership Domain teams are responsible for their data aligning with the boundaries of their team s domain An authorization layer will be required for implementing those boundaries for some team Data as a product Analytical data should be treated as a product with consumers beyond the domain An owner consumer relationship will exist where consumers require access to products owned by a different team Self serve data infrastructure platform A data platform team provides domain agnostic tools and systems to build execute and maintain data products Federated governance Interoperability of data products is achieved through standardization promoted by the governance group Owner consumer relationship A data consumer requests access to some data owned by a different team in a different domain For example a table in a database The data owner grants access by approving the access request Upon the approval of an access request a new permission is added to the specific table Our authorization layer must be able to provide the above capability if we want to implement a data mesh with success Data Lake In this section we will write a brief recap about what we explained in previous article gu martinm how to build your own data platform fff AWS S Notebooks Spark jobs clusters etc etc run in Amazon virtual servers called EC These virtual servers require permissions for accessing AWS S These permissions are given by IAM Roles We will be working with Amazon Web Services As we said before because the amount of data to be stored is huge we can not use HDD or SSD data storages we need something cheaper In this case we will be talking about AWS S Also in order to ease the use of the Data Lake we can implement metastores on the top of it For example Hive Metastore or Glue Catalog We are not going to explain deeply how a metastore works that will be left for another future article When using a notebook for example a Databricks notebook and having a metastore the first thing that the notebook will do is to ask the metastore where the data is physically located Once the metastore responds the notebook will go to the path in AWS S where the data is stored using the permissions given by the IAM Role Direct access or with a metastore We have two options for working with the data With or without using a metastore With the metastore users can have access to the data in the Data Lake in an easier way because they can use SQL statements as they do in any other databases Authorization direct access Consumers run their notebooks or any other applications from their AWS accounts and consume data located in the producer s account These notebooks and applications run in Amazon virtual servers called Amazon EC instances and for accessing the data located in AWS S in the producer s account they use IAM Roles the permissions for accessing the data S bucket policyFor example for being able to access to the S bucket called s producer with the IAM Role with ARN arn aws iam ACCOUNT CONSUMER role IAM ROLE CONSUMER we can use the following AWS S bucket policy in the s producer bucket Direct accessHere we are showing an example where from a Databricks notebook using the above IAM Role and running in the consumer account we are able to access data located in the producer s account Can we do it better With Glue Catalog as metastore data in S can be accessed as if it was stored in a table with rows and columns If we use tables instead of the direct access we can grant permissions even at column level Lake Formation provides its own permissions model that augments the IAM permissions model This centrally defined permissions model enables fine grained access to data stored in data lakes through a simple grant or revoke mechanism much like a database Lake Formation permissions are enforced using granular controls at the column row and cell levels Authorization Lake Formation For using Lake Formation we will need the following elements An application running in some machine in an AWS account For example an AWS EC instance where a Spark notebook will be executed A shared resource between the producer and consumer s account In this case we are sharing the S bucket called producer An IAM Role with permissions for using the producer s bucket Two AWS Glue Catalogues as metastores The one in the consumer s account will be in charge of forwarding the table resolution to the metastore in the producer s account Both metastores are also shared between the two accounts The catalogue in the producer s account contains all the required information for translating the virtual table to its physical S location In the below screenshots you can see the Lake Formation configuration for the Glue metastore located in the producer s account First you can see the table and database where the producer s table is located You can also see that we are sharing the specific table with the consumer s account Database schemaTable producerIn the above table we can configure access permissions For example we can decide that we will be allowing only the use of SELECT statements from the consumer s account and also the only column that will be shown is the one called brand id Now from the Spark notebook running in the consumer s account we can run SQL statements against the table located in the producer s account Because we only allowed access to the column called brand id the consumer will only see values for that column Any other column will be hidden Conclusion In this article we have explained how you can implement an authorization layer using AWS IAM Roles and AWS Lake Formation With this authorization layer we will be able to resolve the following problems Producers and consumers from different domains must have the capability of working in an isolated way if they wish so if we want to implement a data mesh with success Producers must be able to decide how consumers can access their data They are the data owners and they decide how others use their data Fine grained permissions can be established At column and even if we want at row level This will be of great interest if we want to be GDPR compliant More information about how to implement the GDPR in your own data platform will be explained in future articles Stay tuned for the next article about how to implement your own Data Platform with success I hope this article was useful If you enjoy messing around with Big Data Microservices reverse engineering or any other computer stuff and want to share your experiences with me just follow me	2023-06-02 21:43:39
ニュース	BBC News - Home	India train crash: More than 120 dead after Odisha incident	https://www.bbc.co.uk/news/world-asia-india-65793257?at_medium=RSS&at_campaign=KARANGA	odisha	2023-06-02 21:05:59
ニュース	BBC News - Home	Caroline Flack's mum criticises ITV over Phillip Schofield	https://www.bbc.co.uk/news/uk-65793818?at_medium=RSS&at_campaign=KARANGA	caroline	2023-06-02 21:06:47
ニュース	BBC News - Home	Jojo Moyes: 'I've always been a woman's woman'	https://www.bbc.co.uk/news/entertainment-arts-65794240?at_medium=RSS&at_campaign=KARANGA	female	2023-06-02 21:05:57
ニュース	BBC News - Home	British Vogue's Edward Enninful to take on new global role	https://www.bbc.co.uk/news/entertainment-arts-65794311?at_medium=RSS&at_campaign=KARANGA	edward	2023-06-02 21:46:56
ニュース	BBC News - Home	Faith Kipyegon breaks women's 1500m world record as Laura Muir comes second in Diamond League	https://www.bbc.co.uk/sport/athletics/65794409?at_medium=RSS&at_campaign=KARANGA	Faith Kipyegon breaks women x s m world record as Laura Muir comes second in Diamond LeagueFaith Kipyegon sets a new m world record in the Florence Diamond League meeting on Friday with Britain s Laura Muir a distant second	2023-06-02 21:11:29
ビジネス	東洋経済オンライン	岸田政権の｢奇妙な安定性｣を支える2つの要因 ｢アメリカ子会社の社長｣を｢愚民｣が支える構図 | 新競馬好きエコノミストの市場深読み劇場 | 東洋経済オンライン	https://toyokeizai.net/articles/-/677088?utm_source=rss&utm_medium=http&utm_campaign=link_back	東洋経済オンライン	2023-06-03 06:30:00
ビジネス	東洋経済オンライン	算数が苦手な子になる｢親が言いがち｣余計な一言 サピックス小学部に聞いた｢算数と数学｣の違い | 学校・受験 | 東洋経済オンライン	https://toyokeizai.net/articles/-/673464?utm_source=rss&utm_medium=http&utm_campaign=link_back	東洋経済オンライン	2023-06-03 06:30:00
海外TECH	reddit	Astralis vs MOUZ / IEM Dallas 2023 - Quarter-Final / Post-Match Discussion	https://www.reddit.com/r/GlobalOffensive/comments/13yptsz/astralis_vs_mouz_iem_dallas_2023_quarterfinal/	Astralis vs MOUZ IEM Dallas Quarter Final Post Match DiscussionAstralis MOUZ Inferno Ancient Nuke nbsp nbsp Map picks Astralis MAP MOUZ Overpass X X Mirage Inferno Ancient Vertigo X X Anubis Nuke nbsp Full Match Stats Team K D ADR KAST Rating Astralis device blameF Altekz Buzz glave MOUZ torzsi JDC dexter frozen xertioN nbsp Individual Map Stats Map Inferno Team CT T OT Total Astralis T CT OT MOUZ nbsp Team K D ADR KAST Rating Astralis device blameF Altekz Buzz glave MOUZ dexter torzsi JDC frozen xertioN Inferno detailed stats and VOD nbsp Map Ancient Team T CT Total Astralis CT T MOUZ nbsp Team K D ADR KAST Rating Astralis blameF Altekz device Buzz glave MOUZ torzsi xertioN JDC frozen dexter Ancient detailed stats and VOD nbsp This thread was created by the Post Match Team If you want to share any feedback or have any concerns please message u CSGOMatchThreads submitted by u CSGOMatchThreads to r GlobalOffensive link comments	2023-06-02 21:26:01