投稿時間:2022-01-23 01:19:26 RSSフィード2022-01-23 01:00 分まとめ(25件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
IT	気になる、記になる…	Google、｢Boreal｣というコードネームの新型｢Chromecast｣を準備中か	https://taisy0.com/2022/01/23/151074.html	chromecast	2022-01-22 15:58:35
IT	気になる、記になる…	｢Google Pixel 6a｣は5月発表か	https://taisy0.com/2022/01/23/151072.html	maxja	2022-01-22 15:33:09
python	Pythonタグが付けられた新着投稿 - Qiita	Wordle のソルバーを Python で実装する	https://qiita.com/yush1ga/items/3a99fcf4e53c14143595	WordleのソルバーをPythonで実装するWordle戦略※早く実装を見せろという方のためのソルバーの実装はこちらWordleでは序盤は当てずっぽうで単語を入力して情報を得ていき、その情報をもとに後半で正解を狙いに行くというのが一般的な戦略かと思われます。	2022-01-23 00:03:56
js	JavaScriptタグが付けられた新着投稿 - Qiita	ブラウザゲームを作ってみよう(その4)	https://qiita.com/noji505/items/ac21d108add93d2470e6	ブラウザゲームを作ってみようそのはじめにそのではキー入力を行いました。	2022-01-23 00:49:20
js	JavaScriptタグが付けられた新着投稿 - Qiita	typeof と instanceof	https://qiita.com/anko-taro/items/1393f43533bcdae29247	typeofとinstanceof変数sが文字列かどうかを確認するコードで、typeofsstringsinstanceofStringというものを見かけた。	2022-01-23 00:08:49
Ruby	Rubyタグが付けられた新着投稿 - Qiita	Tempfile オブジェクトから Content-Type(mime type) を取得する方法	https://qiita.com/tkm_kj/items/2f190269e9bf9a160782	TempfileオブジェクトからContentTypemimetypeを取得する方法やりたいことTempfileのオブジェクトを作成後、そこからContentTypemimetypeを取得したい。	2022-01-23 00:42:47
AWS	AWSタグが付けられた新着投稿 - Qiita	SES に SMTP リレーをする Postfix サーバーを構築してみた	https://qiita.com/sugimount-a/items/4784439d1275abd4b9cc	環境情報ECInstanceAmazonLinuxPostfixSESを使ってドメインは事前に認証済みPostfixInstallAmazonLinuxの環境では、既にPostfixがインストールされています。	2022-01-23 00:51:53
AWS	AWSタグが付けられた新着投稿 - Qiita	AWS Cloud Practitioner Essential まとめ10	https://qiita.com/bhkbb2/items/f28cac583f355c7cefcf	もしくはほとんど変更しない移行リプラットフォームRiftTinkerShiftアプリケーションの一部に最適化は入れるものの、中核となるアーキテクチャは変更しない。	2022-01-23 00:22:06
Azure	Azureタグが付けられた新着投稿 - Qiita	Azure Blob StorageをNFSでマウント	https://qiita.com/dg4101/items/9c923954e602f4697706	VMからNFSでマウントAzureでUbuntuのVMを使用する場合は、こんな感じでできます。	2022-01-23 00:38:47
Git	Gitタグが付けられた新着投稿 - Qiita	GitでLFS対象ファイルのプル、プッシュができない不具合の解消法	https://qiita.com/ysk2501/items/cea0c6fb7041441c4017	GitでLFS対象ファイルのプル、プッシュができない不具合の解消法最近数日間悩まされていたトラブルが解消したので備忘録としてのメモです。	2022-01-23 00:35:57
Git	Gitタグが付けられた新着投稿 - Qiita	Git で文書管理（下書き中）	https://qiita.com/tama2ka/items/5c0058a5aa400c7b47f4	github	2022-01-23 00:07:23
Ruby	Railsタグが付けられた新着投稿 - Qiita	Tempfile オブジェクトから Content-Type(mime type) を取得する方法	https://qiita.com/tkm_kj/items/2f190269e9bf9a160782	TempfileオブジェクトからContentTypemimetypeを取得する方法やりたいことTempfileのオブジェクトを作成後、そこからContentTypemimetypeを取得したい。	2022-01-23 00:42:47
海外TECH	MakeUseOf	Here’s Why You Shouldn’t Put Stickers on Your MacBook	https://www.makeuseof.com/why-you-shouldn-t-put-stickers-on-macbook/	macbook	2022-01-22 15:30:42
海外TECH	MakeUseOf	10 Unique Ways to Organize Your Android Apps	https://www.makeuseof.com/how-to-organize-your-android-apps/	android	2022-01-22 15:00:45
海外TECH	DEV Community	Microsoft Azure Data Fundamentals Cheat Sheet	https://dev.to/theseregrets/microsoft-azure-data-fundamentals-cheat-sheet-36kp	Microsoft Azure Data Fundamentals Cheat SheetWritten with the collaborative effort of Rishita Shaw and Kumar Rahul This a cheat sheet for Microsoft Certified Exam DP Microsoft Azure Data Fundamentals This is not enough for preparation but it s definitely enough for revision Module Explore core data concepts DataStructured relational databases stored in SQL Server or Azure SQL Database Unstructured audio and video files and binary data files BLOB Semistructured key value stores and graph databases key value database stores data as a single collection without structure or relation Azure cosmodb Access Read only management team of large org read write customers Ownership db admin Data analysts and data managers Data processing Transactional oltp Splitting tables out into separate groups of columns like this is called normalization it can make querying more complex Analytical oatp big picture view of the information held in a databaseRelational dbNon relational dbA transaction is a sequence of operations that are atomic This means that either all operations in the sequence must be completed successfully or if something goes wrong all operations run so far in the sequence must be undone ACID Atomicity Consistency Isolation Durability Many systems implement relational consistency and isolation by applying locks to data when it is updated The lock prevents another process from reading the data until the lock is released The lock is only released when the transaction commits or rolls back Distributed dbeventual consistency Analytical workloadsBatch processing and streamingBatch processing adv and disadv adv large vol scheduled run disadv time delay minor issues can stop the process Diff b w streaming and batch data scope size performance latency analysis Module Explore roles and responsibilities in the world of data Data job roleDatabase Administrators operational aspects of on premises and cloud based database manage databases assign permissions to users implement policies manage security storing backup copies of data and restoring data in case of any failures Data Engineers work with data applying data cleaning routines and ingestion identifying business rules and turning data into useful information Data Analysts explore and analyze data to create visualizations and charts to enable organizations to make informed decisions TASK AND RESPONSIBILITYDb admin Install upgrade db server amp toolsAllocating controlling monitoring and modifying storage Enrolling users and securityBackup restore and archiveGenerate reportsTools SQL Server Management Studio pgadmin mysql workbench azure data studioData eng Developing constructing testing acquisition examining automating and maintaining db and dataDeploying sophisticated analytics programs machine learning and statistical methods improve data reliability efficiency and quality and researchTools Microsoft SQL Server Azure SQL Database Azure Databricks and Azure HDInsight cosmodbData Analyst Creating charts and graphs histograms geographical mapsTransforming improving and integrating dataFinding hidden patterns using data and delivering info by creating rich graphical dashboards and reportsTools excel power bi Module Describe concepts of relational data Relational db You design a relational db by creating a data model The primary key indicates the column or combination of columns that uniquely identify each rowForeign Key are reference or link to the primary key of another table and are used to maintain the relationships between tablesAll data is tabular Entities are modeled as tables each instance of an entity is a row in the table and each property is defined as a column All rows in the same table have the same set of columns A table can contain any number of rows Supports sqlsuited for OLTP applicationsIndexcontains a copy of this data in a sorted order with pointers to the corresponding rows in the table and can create many indexesconsume additional storage space and additional work can slow down operations and incur chargesA clustered index physically reorganizes a table by the index keyViewvirtual table based on the result set of a queryon premises hosting vs cloudIaas installing and configuring the software patching taking backups and restoring data virtual machine in the cloudbest for migrations and applications requiring operating system level accessPaaSDoes not expose the underlying operating system and hardware to your applicationsAzure automatically creates the necessary virtual machines networks and other devices for your requirementQuickly handles this scaling for you Module Explore concepts of non relational data Non relational db enable you to store data in a very flexible mannerdon t impose a schema on data rather focus on the data itselfstore the information for entities in collections or containersEach entity should have a unique key value and are usually stored in key value order advanced non relational systems support indexing Azure Cosmos DB Semi structured data that contains fields defined on a per entity basisJson Avro ORC and ParquetAvro is a row based format Each record contains a header json that describes the structure of the data binary in the record ORC Optimized Row Columnar format organizes data into columns Hive supports SQL like queries over unstructured dataParquet is another columnar data format row group Unstructured data store video and audio data as block blobs in an Azure Storage accountA block blob only supports basic read and write operations No sqlkey value stores key uniquely identifies the item and the value opaque holds the data for the item read and write data very quickly excellent choice for data ingestionAzure Table storage cosmo dbdocument databases each document has a unique ID but the fields in the documents are transparent to the dbmsXML YAML JSON BSON format or plain textenables you to query and filter data by using the values in these fields Some create the document key automatically and support indexing to facilitate fast lookupAzure Cosmos DB implements a document database approach in its Core SQL API column family databases ORC and Parquet filesdenormalized approach to structuring sparse datacolumn family database as holding tabular data comprising rows and columns but you can divide the columns into groups known as column familiesApache Cassandra Azure Cosmos DB supports the column family approach through the Cassandra API graph databases store entities but focuses on the relationships between entitiesnodes entities and edge relationships between nodes efficiently perform queries Azure Cosmos DB supports graph databases using the Gremlin API Module Explore concepts of data analytics Data ingestion Data Processing ELT and ETL SQL Server Integration Services and Azure Data Factory Azure Data Factory is a cloud based data integration service that allows you to create data driven workflows for orchestrating data movement and transforming data at scale ETL processes that transform data visually with data flows or by using compute services such as Azure HDInsight Hadoop Azure Databricks and Azure SQL Database ReportingBusiness Intelligence BI Benchmarking Comparison with other companies in the same industry Data VisualizationMost famous tool in Azure is Power BI for data visualization you can connect to multiple different sources of data and combine them into a data modelBar and column chart Line chart Matrix Key influencer Tree map ScatterFiled mapData analytics Descriptive what happened By developing KPIs Key Performance Indicators these strategies can help track the success or failure of key objectivesDiagnostic why happened Predictive what will happen in the future using neural networks decision trees and regression Prescriptive what action should we take to achieve a goal Cognitive Cognitive analytics helps you to learn what might happen if circumstances change and how you might handle these situations It uses several NLP Natural Language Processing concepts to make sense of previously untapped data sources such as call center conversation logs and product reviews Module Explore relational data services in Azure Data base Stored procedure A stored procedure is a block of code that runs inside your database A linked server is a connection from one database server to another SQL Server can use linked servers to run queries on one server that can include data retrieved from other servers these are known as distributed queries IAAS Infrastructure as a service e g azure virtual network PAAS Platform as a service e g Azure SQL Databases SAAS Software as a Service e g office Azure Data Services Azure Data Services fall into the PaaS category These services are a series of DBMSs managed by Microsoft in the cloud Each data service takes care of the configuration day to day management software updates and security of the databases that it hosts All you do is create your databases under the control of the data service Most famous database service is Azure SQL databaseAzure Database for sql server MariaDB server Postgre SQL server Microsoft also provide services for non relational dbms such as cosmos DBAzure Data Services ensure that your databases are available for at least of the time There are cost associated with the running database in azure data servicesCan t shutdown the database and restart it later This services are always on SQL Server on Azure Virtual Machines SQL Server on Virtual Machines enables you to use full versions of SQL Server in the Cloud without having to manage any on premises hardware SQL Server running on an Azure virtual machine effectively replicates the database running on real on premises hardware Migrating from the system running on premises to an Azure virtual machine lift and shift refers to the way in which you can move a database directly from an on premises server to an Azure virtual machine without requiring that you make any changes to it A hybrid deployment is a system where part of the operation runs on premises and part in the cloudAzure SQL Database Azure SQL Database is a PaaS offering from Microsoft You create a managed database server in the cloud and then deploy your databases on this server Single Database Elastic Pool database This option is similar to Single Database except that by default multiple databases can share the same resources such as memory data storage space and processing power through multiple tenancy Managed instance Azure SQL Database is often used for Modern cloud applications that need to use the latest stable SQL Server features Applications that require high availability Systems with a variable load that need the database server to scale up and down quickly SQL Database helps secure your data by providing encryption For data in motion it uses Transport Layer Security For data at rest it uses Transparent Data Encryption For data in use it uses Always Encrypted Azure SQL Database Managed Instance You have complete control over this instance much as you would for an on premises server The Managed instance service automates backups software patching database monitoring and other general tasks but you have full control over security and resource allocation for your databases Managed instances depend on other Azure services such as Azure Storage for backups Azure Event Hubs for telemetry Azure Active Directory for authentication Azure Key Vault for Transparent Data Encryption TDE and a couple of Azure platform services that provide security and supportability features The managed instances make connections to these services MySQL MariaDB compatibility with oracle database One notable feature of MariaDB is its built in support for temporal data A table can hold several versions of data enabling an application to query the data as it appeared at some point in the past PostgreSQLAzure Database for MySQL High availability features built in Predictable performance Easy scaling that responds quickly to demand Secure data both at rest and in motion Automatic backups and point in time restore for the last days Enterprise level security and compliance with legislation Azure Database for MariaDB Built in high availability with no additional cost Predictable performance using inclusive pay as you go pricing Scaling as needed within seconds Secured protection of sensitive data at rest and in motion Automatic backups and point in time restore for up to days Enterprise grade security and compliance Azure Database for PostgreSQL same property as azure database for my sql Azure Database for PostgreSQL single server Each tier supports different numbers of CPUs memory and storage sizesAzure Database for PostgreSQL Hyperscale Citus Data is split across nodesUse Azure database migration service to migrate on premise Mysql mariaDB or postgresql to a database running the corresponding data services in Azure Module Explore provisioning and deployment in database service in azure What is Provisioning Provisioning is the act of running a series of tasks that a service provider such as Azure SQL Database performs to create and configure a service The act of increasing or decreasing the resources used by a service is called scaling Tools to provision services Azure portal Display list of service specific pages before actual provisioning Azure command line interface CLI basic command prompt and powershell command we run on windows to automate service creation Azure PowershellAzure Resource manager templates describes the service or services that you want to deploy in a text file in a format known as JSONProvisioning in Azure SQL databaseProvisioning PostgreSQL and MySQL Hyperscale option for postgreSQL supports Horizontal Scaling Query parallelization Excellent support for multi tenant applications real time operational analytics and high throughput transactional workloadsConfiguring relational data servicesConfigure connectivity and firewall Configure connectivity to virtual networks and on premises computersAzure SQL Database communicates over port A firewall rule of enables all Azure services to pass through the server level firewall rule and attempt to connect to a single or pooled database through the server Configure connectivity from private endpoints Configure Authentication Configure access control who or what can access your resources Role assignment consist of three elements Security principleRole definition Collection of permission Owner Contributor Reader User access administrator Scope lists set of resources that the access applies to You add role assignments to a resource in the Azure portal using the Access control IAM pageConfigure advance data security Configure Azure SQL DatabaseAn ACL access control list contains a list of resources and the objects users computers and applications that are allowed to access those resourcesConnectivity from within AzureConnectivity from outside azure Configure DoS Denial of Service guard DoSGuard actively tracks failed logins from IP addressesConfigure Azure Database for PostgreSQL Connections to your Azure Database for PostgreSQL server communicate over port Configure read replicas replicate data from an Azure Database for PostgreSQL server to a read only serverConfigure Azure Database for MySQLConfigure server parameter Configure read replicas same as postgreSQL Module Query relational data in azure SQL structured query languageTransact SQL T SQL This version of SQL is used by Microsoft SQL Server and Azure SQL Database pgSQL This is the dialect with extensions implemented in PostgreSQL PL SQL This is the dialect used by Oracle PL SQL stands for Procedural Language SQL Data Manipulation Language DML SELECT INSERT UPDATE DELETE Data Definition Language DDL CREATE ALTER DROP RENAMEQuery relational data in Azure SQL DatabaseRetrieve connection information for azure sql DB Use the azure portal to query a DBUse SQLCMD to query a databaseUSe Azure data studio Azure Data Studio is a graphical utility for creating and running SQL queries from your desktopSQL server management studioUse SQL Server Data Tools in Visual StudioQuery relational data in Azure Database for PostgreSQLRetrieve connection information for Azure Database for PostgreSQLUse psql to query a databasepsql commands include l to list databases dt to list the tables in the current database Connect to PostgreSQL database using Azure Data StudioQuery relational data in Azure Database for MySQLport Module Explore non relational data offerings in Azure Azure Table Storage items are referred to as rows must have a key and fields are known as columnsstore semi structured data schemaless Denormalized datasplits a table into partitions Partitioning is a mechanism for grouping related rows based on a common property or partition keyPartitions are independent from each other you can include the partition key in the search criteriaItems in the same partition are stored in row key order quickly perform Point queries that identify a single row and Range queries that fetch a contiguous block of rows in a partitionThe columns in a table can hold numeric string or binary data up to KB in size A table can have up to columns apart from the partition and row keys The maximum row size is MBAdv no need to map and maintain the complex relationshipsFast operations like insertion deletion querySimple to scaleStoring TBs of structured data capable of serving web scale applicationsStoring datasets that don t require complex joins foreign keys or stored proceduresCapturing event logging and performance monitoring dataDisadvConsistency isn t guaranteedno referential integritydifficult to filter and sort on non key dataThe data for each table is replicated three times within an Azure region For increased availabilitytransparently switch to a working replica while the failed replica is recoveredconfigure security and role based access controlAzure Blob storageunstructured data or blobsyou create blobs inside containersBlock blobs handled as a set of blocks used to store discrete large binary objects that change infrequently Page blobs organized as a collection of fixed size byte pages support random read and write operations used to implement virtual disk storage for virtual machinesAppend blobs a block blob optimized to support append operations updating or deleting existing blocks isn t supportedAcces tiersThe Hot tier is accessed frequently data is stored on high performance media The Cool tier has lower performance accessed infrequently The Archive tier provides the lowest storage cost but with increased latency for historical data that mustn t be lost but is required only rarely stored in an offline state A lifecycle management policy can automatically move a blob from Hot to Cool and then to the Archive tier as it ages and is used less frequentlyServing images or documents directly to a browserStoring data for backup and restore disaster recovery and archiving analysis and distributed accessVersioningSoft deleteSnapshotsChange FeedAzure File storageCreate and access files shares in the cloudexposes file shares using the Server Message Block SMB protocoltools such as the AzCopy utility or azure portalServices standard and premium greater throughput Migrate existing applications to the cloud access data using file based APIs Share server data across on premises and cloud leverage the availability durability scalability and geo redundancy built into the Azure storage platform Integrate modern applications with Azure File Storage rest api Simplify hosting High Availability HA workload dataAzure Cosmos DBmulti model NoSQL dbmsmanages data as a partitioned set of documents collection of fields identified by a key use JSONprovides APIs that enable you to access these documentsSQL API provides a SQL like query language over documentsTable API use the Azure Table Storage API to store and retrieve documentsMongoDB API enable a MongoDB application to run unchanged against a Cosmos DB databaseCassandra API column family database management system provides a Cassandra like programmatic interface for Cosmos DBGremlin API implements a graph database interface to Cosmos DBUnlike Azure Table storage documents in a Cosmos DB partition aren t sorted by ID Instead Cosmos DB maintains a separate index This index contains not only the document IDs but also tracks the value of every other field in each documentall databases are replicated within a single regionreplication is transparent and failover from a failed replica is automaticguarantees less than ms latencies for both reads indexed and writes at the th percentile all around the worldall data in Cosmos DB is encrypted at rest and in motionModule Explore provisioning and deploying non relational data services in Azure provisioning non relational data servicesProvisioning is the act of running a series of tasks that a service provider it is opaque Parameters initially set can be later modified ToolsThe Azure portalThe Azure command line interface CLI run command using cloud shellAzure PowerShellAzure Resource Manager templates deploy in json Provision Azure Cosmos DBAzure CLI Azure PowerShell or an Azure Resource Manager templateDatabases and containers are the primary resource storage space consumersuses the concept of Request Units per second RU s to manage the performance and cost of databasesIf you underprovision by specifying too few RU s Cosmos DB will start throttling performanceconfiguring non relational data servicesYou can connect to these services from an on premises network the internet or from within an Azure virtual networkAzure Private Endpoint is a network interface that connects you privately and securely to a service powered by Azure Private LinkPrivate endpoint connections page for a service allows you to specify which private endpoints if any are permitted access to your serviceAzure Active Directory Azure AD provides superior security and ease of use over access key authorizationConfigure access controlAzure role based access controlAllow one user to manage virtual machines in a subscription and another user to manage virtual networks Allow a database administrator group to manage SQL databases in a subscription Allow a user to manage all resources in a resource group such as virtual machines websites and subnets Allow an application to access all resources in a resource group security principal is an object that represents a user group service or managed identity that is requesting access to Azure resources role definition often abbreviated to role is a collection of permissionsOwnerContributorReaderUser Access Administratorscope lists the set of resources that the access applies toTools Access control IAM Configure consistency in cosmo dbEventual least consistent Changes won t be lost they ll appear eventually changes could appear out of order lowest latency and least consistencyConsistent Prefix ensures that changes will appear in order although there may be a delay before they become visibleSession If an application makes a number of changes they ll all be visible to that application and in orderBounded Staleness There s a lag between writing and then reading the updated dataStrong In this case all writes are only visible to clients after the changes are confirmed as written successfully to all replicasConfigure shared access signaturesuse shared access signatures SAS to grant limited rights to resources in an Azure storage account for a specified time periodaccess resources such as blobs and files without requiring that they re authenticated firstModule Manage non relational data stores in Azure data operations in Cosmos DBData Explorer in the Azure portal to run ad hoc queriesCosmos DB Data Migration tool to perform a bulk load or transfer of data Azure Data Factory to import data from another sourcecustom application that imports data using the Cosmos DB BulkExecutor library make use of multiple concurrent threads to batch your data into chunks and load the chunks in parallel Create your own application that uses the functions available through the Cosmos DB SQL API client libraryinsufficient throughput capacity configured results in https error Query Azure Cosmos DBThe Cosmos DB SQL API supports a dialect of SQL for querying documents using SELECT statementsManage Azure File storage Module Examine components of a Modern data warehouse The process of combining all of the local data or gathering data from many different sources within an organization is known as data warehousing The process of analyzing streaming data and data from the Internet is known as Big Data analytics Azure Synapse Analytics combines data warehousing with Big Data analytics online analytical processing OLAP Modern data Warehousing It contains a mixture of relational and non relational data like media files social media streams or internet of things Tools we can use Azure Data Factory Azure Data Lake Storage Azure Databricks Azure Synapse Analytics and Azure Analysis Services power BI is used to analyze and visualize the data generating reports and charts gathers data from many different sources within an organization provide answers to complex queries unlike a traditional relational databasehandle big datacontain a mixture of relational and non relational data including files social media streams and Internet of Things IoT sensor dataCombine batch and stream process upto the second data and historical data are important for a company Upto the second data is generated by steam process and historical data is generated by batch process Azure Data factory its a data integration service Purpose of the azure data factory is to retrieve data from one or more data sources and to convert it into the format that you process Work performed by azure data factory is defined as pipeline of operations Azure Data Lake storage It s a repository for Large Quantities of Raw data Data is fast load and processed because its raw amp unprocessed A data warehouse holds structured information but a data lake stores raw data It s an extension of Azure Blob storage It organizes files into directories and sub directories but blob storage only mimics a directory structure It supports POSIX Portable Opening system interface and compatible with HDFS Hadoop distributed file system Azure DataBricks Azure Databricks is an Apache Spark environment running on Azure to provide big data processing streaming and machine learning Azure Databricks provides a graphical user interface where you can define and test your processing step by step before submitting it as a set of batch tasks Azure Synapse Analytics It s an analytical engine Design to process large data very quickly Ingest data from external sources and then transform and aggregate this data into a format suitable for analytics processing its used to process data which we have read in and processed locally Control node is the brain of architecture like a front end which interacts with all applications When we submit a process request it transforms it into smaller requests that run against distinct subsets of the data in parallel We have to submit queries in Transact SQL statementsPolyBase enables you to retrieve data from relational and non relational sources such as delimited text files Azure Blob Storage and Azure Data Lake Storage Azure Analysis Service Azure Analysis Services enables you to build tabular models to support online analytical processing OLAP queries You can combine data from various sources e g lake analytics sql db cosmos db Difference between Analysis service and synapse analytics Use Azure Synapse Analytics for Very high volumes of data multi terabyte to petabyte sized datasets Very complex queries and aggregations Data mining and data exploration Complex ETL operations ETL stands for Extract Transform and Load and refers to the way in which you can retrieve raw data from multiple sources convert this data into a standard format and store it Low to mid concurrency users or fewer Use Azure Analysis Services for Smaller volumes of data a few terabytes Multiple sources that can be correlated High read concurrency thousands of users Detailed analysis and drilling into data using functions in Power BI Rapid dashboard development from tabular data Azure HDInsight Azure HDInsight is a big data processing service that provides the platform for technologies such as Spark in an Azure environment Hadoop is an open source framework that breaks large data processing problems down into smaller chunks Hive is a SQL like query facility that you can use with an HDInsight cluster to examine data held in a variety of formats Module Explore large scale data analytics Ingest data using Azure Data FactoryHTAP Hybrid transactional analytical processing analyze operational data into original location Orchestration is the process of directing and controlling other services and connecting them together to allow data to flow between them A linked service provides the information needed for Data Factory to connect to a source or destinationA dataset in Azure Data Factory represents the data that you want to ingest input or store output If the data is structured the data set specifies how data is structured A pipeline is a logical grouping of activities that together perform a task The activities in a pipeline define actions to perform on your data It includes a lot of activity like looping data in for each loop can use if condition to execute the activity conditionally can map input into different formats as per output is required A trigger enables you to schedule a pipeline to occur according to a planned scheduleIngest data using PolyBasePolybase is a feature of SQL server amp azure synapse analytics that enables you to run Transact SQL queries that read data from external data sources It makes external data appear like tables in a sql db Azure SQL Database does not support PolyBase Ingest data using SQL Server Integration ServicesSQL Server Integration Services SSIS is a platform for building enterprise level data integration and data transformations solutions It is used to solve complex business problems SSIS can extract and transform data from a wide variety of sources such as XML data files flat files and relational data sources and then load the data into one or more destinations Module Get started building with Power BIcollection of software services apps and connectors that work together to turn your unrelated sources of data into coherent visually immersive and interactive insights Power BI Desktop an online SaaS Software as a Service Use Power BIBring data into Power BI Desktop and create a report Publish to the Power BI service where you can create new visualizations or build dashboards Share dashboards with others especially people who are on the go View and interact with shared dashboards and reports in Power BI Mobile apps Building blocks of Power BIVisualizationsDatasetsReportsDashboardTilesapp is a collection of preset ready made visuals and reports that are shared with an entire organization Power BI can just as easily connect to a whole assortment of software services also called SaaS providers or cloud services Salesforce Facebook Google Analytics and more	2022-01-22 15:51:07
海外TECH	DEV Community	Best Animation Libraries for ReactJS	https://dev.to/surajondev/best-animation-libraries-for-reactjs-2h34	Best Animation Libraries for ReactJS IntroductionReactJS is the most popular and loved framework among web developers for developing the frontend From the launch it has seen only growth There are tons of libraries on the web for ReactJS that made React more awesome Today we are going to look into some of the best animation libraries that will help you to create animation These libraries are easy to install learn and create animation So let s get started Framer MotionA production ready motion library for React Utilize the power behind Framer the best prototyping tool for teams Proudly open source Installationnpm i framer motion Usage Importsimport motion from framer motion CodeIn a motion div tag you define the animation of the component animate You define the properties of the animation here These are CSS properties transition You define the transition properties here Such as repeat duration etc lt motion div animate x borderRadius transition repeat Infinity duration className circle gt CodeSandbox Example React Springreact spring is a spring physics based animation library that should cover most of your UI related animation needs Installationnpm i react spring Usage Importsimport useSpring animated from react spring CodeWe have imported two things useSpring and animated useSpring We define all the properties of the animation and transition here from define the initial properties of the element and to is the final properties animate It is used to define the element for the transition It works as a component that takes style as the defined animation in useSpring const props useSpring to y opacity from y opacity lt animated div style props className circle gt lt animated div gt CodeSandbox Example React MotionA spring that solves your animation problems Installationnpm i react motion Usage Importsimport Motion spring from react motion CodeMotion Every animation and property is wrapped inside the Motion component It contains the default style and animation properties using spring lt Motion defaultStyle translateX opacity style translateX spring opacity spring gt interpolatedStyles gt lt div style transform translateY interpolatedStyles translateX px opacity interpolatedStyles opacity className circle gt lt div gt lt Motion gt CodeSandbox ExampleRefersh to play the animation Last NoteThese are Animation libraries for ReactJS that I like and use in my project for animation I hope you will try to use one of the libraries in your next project Thanks for reading the post	2022-01-22 15:31:18
海外TECH	DEV Community	Create Ozark's title animation with Greensock (GSAP)	https://dev.to/robole/create-ozarks-title-animation-with-greensock-gsap-166k	Create Ozark x s title animation with Greensock GSAP To mark the occasion of the TV show Ozark releasing its fourth season I created the title sequence as a web animation For some viewers a show s title sequence is nothing more than a minute long segment to be skipped past or used as an opportunity to squeeze in a quick activity before the show starts but they are missing out in the case of Ozark The show is known for its dynamic plot and stark imagery and if you pay close attention to the show s brief title sequence you can appreciate how it complements the tone of the show It can be interesting to see how the internet dissects these kind of things The first part of the title sequence is extremely simple It s just the word Ozark in a regular font a font so bland it led to Redditors to question the motives behind the choice In a thread entitled Does the Ozark opening sequence font bother anyone else user Suspicious Earth asked the question Considering that Ozark is an incredibly well made show with exceptionally great acting writing world building and directing does the fact that the opening title credits use the MOST basic and generic font bother anyone else I won t go there The bit I like is the second part of the sequence which introduces a large circle with symbols forming inside Upon first watch the symbols that pop up in the middle of the circle may seem arbitrary but after watching the episode you can tie the symbols you see to themes images and plot devices of that episode Some are straightforward and depict something the viewer will literally see later on while others require some creative thinking to work out Here is my animation of Season Episode s title card You can click on it to replay if you missed it There is something else about the symbols that you may not have realized The symbols form the outline of letters which spell out the name of the show The big circle is the letter O and inside you have Z A R K And I recerntly discovered but I haven t tried that if you turn on audio subtitles while watching the title sequence on Netflix it tells you exactly what the symbols are If you want to see an image of the title cards for each episode for the first seasons along with a plot summary you can check out this article Here s What The Symbols in Ozark s Hypnotizing Opening Credits Mean Ok enough chatter let s look at how to make the animation The codeI used Greensock GSAP for the animation You could pull it off with CSS too but I chose GSAP because I wanted an excuse to play with it I drew the image for Season Episode title card as a SVG in Inkscape When I was happy with its appearance I inspected the SVG code and removed any transform styles applied to elements It can mess up animations when multiple transforms are applied to the same element In theory the animation is quite simple you are mostly just fading in the elements at different times and at different rates The outline of the sequence of events is Fade in the circle slowly Begin extending the vertical line Followed quickly by extending the horizontal line After the lines are approxmiately of their complete length fade in the first symbol When the first symbol has reached about opacity fade in the second symbol and follow the same pattern with the next symbols It takes some trial and error to find the right duration and eases for each part GSAP has an ease visulizer that is helpful to narrow the search down We will animate the opacity of the circle and symbols and we will animate transform scale for the lines We animate scaleY for the vertical line and scaleX for the horizontal line This is not intended to be a tutorial on GSAP you can jump into the Getting Start with GSAP guide if you are newcomer However there are a few bits I will point out that may help you when using GSAP for similar type of animations In CSS we set the opacity to zero for all our elements o letter line symbol opacity For the lines we set the initial properties in JavaScript that we want to animate this ensures you get the most accurate result In the past I set these properties in CSS and it can led to erratic results sometimes This is what they recommend tl set vertical line transformOrigin center scaleY tl set horizontal line transformOrigin center scaleX We are create a timeline and use the to function to perform the individual animations I will just focus on the animation of the symbols here as they are the most complicated It is helpful to extract out common values as variables I found it a bit tricky to get the duration ease and the overlapping of the animations the way I wanted So having variables for this is the way to make adjustments simpler let tl new TimelineMax let symbolRevealDuration let symbolRevealOverlap let symbolEase slow false other stuff for animating circle and lines symbolstl to symbol duration symbolRevealDuration ease symbolEase opacity tl to symbol duration symbolRevealDuration ease symbolEase opacity symbolRevealOverlap tl to symbol duration symbolRevealDuration ease symbolEase opacity symbolRevealOverlap tl to symbol duration symbolRevealDuration ease symbolEase opacity symbolRevealOverlap You can see with the to function the third parameter is the position position Number String default ーcontrols the insertion point in the timeline by default it s the end of the timeline By default animations are added to the end of the timeline so that everything is sequenced one after the other We can use the position parameter to overlap the revealing of the symbols For example for symbol we supply which will begin the animation seconds before the end of the timeline This is what I hit upon that worked the way I wanted You could also use lt for the position parameter to start an animation second past the start of the previous animation This is probably simpler to understand as sequence of animations Since I discovered this notation late I didn t feel like redoing it sorry These are some of the lessons you learn along the way I put the code on GitHub along with some other title sequences if you want to grab it That s it I hope you enjoyed this short whistle tour If you a fan of the show enjoy the new season	2022-01-22 15:22:50
海外TECH	DEV Community	How to use Proof-of-Competence Web3 on-boarding framework	https://dev.to/yakult/how-to-use-proof-of-competence-web3-on-boarding-framework-1853	How to use Proof of Competence Web on boarding frameworkWeb is open and one s footprints can be verified on chain Proof of Competence web front end framework provides a good start to building DAPPs to help users measure their own web activities It is developed by Wesley wslyvh He explains it as follows Proof of Competence PoC is an open permissionless framework to create on boarding journeys into the Web space It uses a pluggable task system which can verify that on chain actions have occurred This allows to build up reputation or social DAO scores that proof an address is familiar with the specified technologies or protocols You can find Proof of Competence front end framework at How it worksUsers can connect wallet via MetaMask or WalletConnect to Proof of Competence PoC DAPP to track their own web footprint You get a score for a specific web journey PoC is designed with three concepts Journey a list of tasks for web users stored in JSON format It will give user a score according to the activities of his address Task a unique task such as having Token NFT deployed contract and etc Verifier verify a task using public information e g on chain Subgraphs or public blockchain explorers To use PoC framework we need to write verifier for our own task in the src verifers PoC framework also provides restful API using Next js API routers so that we can use the scores via API PoC API restful API using Next js PoC framework uses on chain data through Infura API Alchemy API Etherscan API etc It is developed with these Node js and web components Next js the React framework by Vercel Chakra UI modular React UI Components Ethers js library to interact with Ethereum Web react React UI for connecting wallect How to install it locallyDownload the project sourcecode git clone git github com wslyvh proof of competence gitThere are sample journeys tasks and verifiers in web src Go to web directory and run this Next js project locally npx next devYou can view the DAPP at http localhost in your browser Add your own web journey with tasksLet s copy web src journeys useWeb json to web src journeys basicWeb json and edit our journey based on it Basic Web User journey has two tasks have address have ENS name Basic Web User version description Onboarding new developers into the Web space website twitter fjun tasks name Have a wallet address description You need a wallet address to enter Web universe points verifier active address name ENS Ethereum Name Service description Register your ENS name at with a reverse lookup points verifier ens reverse lookup You can view our journey at http localhost basicWebThere are several verifiers in this project and we use active address ens reverse lookup has poap active addressfirst transactionens reverse lookuphas poaphas nftdeployed contractvote on snapshotThere are also several dummy verifiers to start with test always falsetest always gtruetest random numberWe can get the journey and score from restful API at http localhost api journey name basicWebhttp localhost api journey score journey basicweb amp address xNote You can also deploy this project online using GitHub amp Vercel Vercel s tutorial We will continue to develop our own task and verifier locally Some preparations for development Add env localWe will use NEXT js environment variables to store infura Alechemy and Etherscan API ID keys Variables with prefix NEXT PUBLIC can be accessed from the browser More details about NEXT js environment variables at Add file env local This file should be added to gitignore if you opensource your project env localNEXT PUBLIC ALCHEMY API URL https eth mainnet g alchemy com v NEXT PUBLIC ALCHEMY API KEY C NEXT PUBLIC INFURA API ID d NEXT PUBLIC ETHERSCAN API KEY ab Notes on Alchemy APIWe will use Alchemy API to interact with Ethereum mainnet Documents can be found at Token APIWe will call alchemy getTokenBalances to get ERC token balance Reference alchemy gettokenbalancesChain API Ethereum APIWe will call eth getBalance to get Ether balance of an address Reference Using Ethereum API we can interact with the Ethereum RPC directly Another Option is to interact with Ethereruem RPC using ethers js through Alechemy endpoint We will use this option to check if a user owns a specific NFT later NFT APIIn the sample NFT verifier in the project it calls Alchemy NFT API getNFTs to get information about the NFTs of an address Two notes here First NFT API is in the V of Alchemy API We need to change the endpoint from V to V should be Second NFT API need to apply manually otherwise your call will return Eth NFT API v not enabled on this account please visit to request access I applied using this form But I haven t received permission yet So we will interact directly with an ERC contract using ethers js in section Add a verifier to measure ETH of your address Step add verifierAdd a verifier called has ETH Let s begin with copy test always true directory and change the directory name to has ETH The index ts in this directory looks like import Task from types export async function verify task Task address string Promise lt boolean number gt return true Step add task to the journeyIn web src journeys basicWeb json add a task name has ETH description Own ETH points verifier has ETH params amount Explanation of this task Points pointsVerifier has ETHparams If the user s address has more than ETH the user gets points Let s go to the web page http localhost basicWeb you can see that you get points as the verifier always returns true now Step program has ETH verifierLet s program the has ETH verifier verifiers has ETH index ts Reminder add in env local NEXT PUBLIC ALCHEMY API URL NEXT PUBLIC ALCHEMY API KEYimport Task from types export async function verify task Task address string Promise lt boolean number gt if address return false try const amount task params amount const url process env NEXT PUBLIC ALCHEMY API URL process env NEXT PUBLIC ALCHEMY API KEY const body jsonrpc method eth getBalance params address latest id const response await fetch url method POST body JSON stringify body const data await response json if data error return false if data result e gt amount return true return false catch e return false We call eth getBalance using Alechmy Etherum API If result gt amount the verifier returns true In my test address there is about ETH in it The task is passed and I get points Let s change the amount in the journey to ETH The task is failed and I don t get points TODO NOTE We program this example quickly and we should consider modifying this verifier so that it can be used in testnet Ropsten sidechain Polygon L Arbitrum Optimism and other EVM Add a verifier to measure ERC tokenIn this section you will add a verifier has token ERC to measure ERC token in your address Add directory has token RRC and then edit index ts verifiers has token ERC index ts Reminder add in env local NEXT PUBLIC ALCHEMY API URL NEXT PUBLIC ALCHEMY API KEYimport Task from types export async function verify task Task address string Promise lt boolean number gt if address return false try const contractAddress task params addressERC const amount task params amount const url process env NEXT PUBLIC ALCHEMY API URL process env NEXT PUBLIC ALCHEMY API KEY const body jsonrpc method alchemy getTokenBalances params address contractAddress id const response await fetch url method POST body JSON stringify body const data await response json if Array isArray data result tokenBalances return false const balance data result tokenBalances if balance error return false if balance tokenBalance e gt amount return true return false catch e return false Let s add a task to measure whether the user s address has ERC ENS Token and the amount should be ENS ERC Token contract address xCDFAbecEaCeFDAdd a task in basicWeb json name has ENS Token description Own ENS ERC Token points verifier has token ERC params addressERC xCDFAbecEaCeFD amount As I have ENS ERC token in my test address I can pass this task and get points Add a verifier to measure ERC NFTIn this section you will add a verifier has nft ERC to check where you have at least one specific ERC token in your address We will use ethers js to call an ERC contract function balanceOf address to check Add has nft ERC index ts has nft ERC index ts Reminder add in env local NEXT PUBLIC ALCHEMY API URL NEXT PUBLIC ALCHEMY API KEYimport Task from types import ethers from ethers import AlchemyProvider from ethersproject providers export async function verify task Task address string Promise lt boolean number gt if address return false const contractAddress string task params addressERC toString if task params addressERC ethers utils isAddress contractAddress return false try const provider new AlchemyProvider task chainId process env NEXT PUBLIC ALCHEMY API KEY const contract await new ethers Contract contractAddress abi provider const balanceOfNFT await contract balanceOf address if balanceOfNFT gt return true return false catch e console log e return false const abi constant true inputs internalType address name owner type address name balanceOf outputs internalType uint name type uint payable false stateMutability view type function Explanation of what we do in this verifier connect to a ERC contract with ethers Contract contractAddress abi provider call contract balanceOf address to get NFT amount of user s address we provide ABI segment of the need contract function balanceOf in const abi Add a task in basicWeb json name Own a ERC NFT description Own an ENS NFT ERC points verifier has nft ERC params addressERC xfabfbfcdffdbaccafea xfabfbfcdffdbaccafea is the contract address of ENS ERC NFT The End of the beginningNow you can begin to add tasks and verifiers to your protocol for on boarding web users	2022-01-22 15:16:41
Apple	AppleInsider - Frontpage News	Apple signs multi-year live-action film deal with Skydance Media	https://appleinsider.com/articles/22/01/22/apple-signs-multi-year-live-action-film-deal-with-skydance-media?utm_medium=rss	Apple signs multi year live action film deal with Skydance MediaSkydance Media is extending its relationship with Apple TV by signing a new multi year deal to develop and produce a series of Apple Original Films In February Apple signed a deal with Skydance in relation to some of its animated projects Almost one year later another deal has been struck this time dealing with live action content The agreement means Skydance Media will be working to develop and produce a series of films that will be released through Apple Original Films reports Deadline Few details about the deal have been revealed though it will apparently be a multi year affair between the two companies Read more	2022-01-22 15:34:32
ニュース	BBC News - Home	New Cross Fire: Artwork to commemorate 14 young people who died	https://www.bbc.co.uk/news/uk-england-london-60084529?at_medium=RSS&at_campaign=KARANGA	arson	2022-01-22 15:01:15
ニュース	BBC News - Home	Search for monkey missing after US motorway crash	https://www.bbc.co.uk/news/world-us-canada-60095057?at_medium=RSS&at_campaign=KARANGA	pennsylvania	2022-01-22 15:30:39
ニュース	BBC News - Home	Everton lose to Villa in Ferguson's first game since Benitez sacking	https://www.bbc.co.uk/sport/football/60005894?at_medium=RSS&at_campaign=KARANGA	Everton lose to Villa in Ferguson x s first game since Benitez sackingEmiliano Buendia scores the winner for Aston Villa as Duncan Ferguson s second spell in interim charge of Everton begins with defeat at Goodison Park	2022-01-22 15:09:58
ニュース	BBC News - Home	Scotland's Jamieson takes one-shot lead into final round in Abu Dhabi	https://www.bbc.co.uk/sport/golf/60097066?at_medium=RSS&at_campaign=KARANGA	championship	2022-01-22 15:32:56
北海道	北海道新聞	石狩管内中心に大雪　ＪＲ２１７本運休　新千歳１１便欠航	https://www.hokkaido-np.co.jp/article/636663/	冬型の気圧配置	2022-01-23 00:18:43
海外TECH	reddit	Lio Rush announces he’s a free agent	https://www.reddit.com/r/SquaredCircle/comments/sa4o6t/lio_rush_announces_hes_a_free_agent/	Lio Rush announces he s a free agent submitted by u BeatsByDrSpi to r SquaredCircle link comments	2022-01-22 15:19:08