投稿時間:2021-10-30 02:37:14 RSSフィード2021-10-30 02:00 分まとめ(45件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
AWS	AWS Big Data Blog	Now Available: Updated guidance on the Data Analytics Lens for AWS Well-Architected Framework	https://aws.amazon.com/blogs/big-data/now-available-updated-guidance-on-the-data-analytics-lens-for-aws-well-architected-framework/	Now Available Updated guidance on the Data Analytics Lens for AWS Well Architected FrameworkNearly all businesses today require some form of data analytics processing from auditing user access to generating sales reports For all your analytics needs the Data Analytics Lens for AWS Well Architected Framework provides prescriptive guidance to help you assess your workloads and identify best practices aligned to the AWS Well Architected Pillars Operational Excellence Security Reliability …	2021-10-29 16:47:19
AWS	AWS for SAP	Licensing options for Microsoft Windows Server/ SQL Server-based SAP workloads on AWS	https://aws.amazon.com/blogs/awsforsap/windows-sql-server-licensing-for-sap-on-aws/	Licensing options for Microsoft Windows Server SQL Server based SAP workloads on AWSIntroduction Customers have been running Microsoft Workloads on Amazon Web Services AWS for over years longer than any other cloud provider AWS has been running SAP workloads since which is also meaningfully longer than any other cloud provider Today more than active customers are running SAP on AWS and many of them …	2021-10-29 16:51:48
Google	Official Google Blog	Pixel art: How designers created the new Pixel 6 colors	http://feedproxy.google.com/~r/blogspot/MKuf/~3/HMg5daWv5G8/	Pixel art How designers created the new Pixel colorsDuring a recent visit to Google s Color Material and Finish better known as CMF studio I watched while Jess Ng and Jenny Davis opened drawer after drawer and placed object after object on two white tables A gold hoop earring a pale pink shell ーall pieces of inspiration that Google designers use to come up with new colors for devices including the just launched Pixel and Pixel Pro “We find inspiration everywhere Jenny says “It s not abnormal to have a designer come to the studio with a toothbrush or some random object they found on their walk or wherever The CMF team designs how a Google device will physically look and feel “Color material and finish are a big part of what defines a product Jess a CMF hardware designer says “It touches on the more emotional part of how we decide what to buy And Jenny CMF Manager for devices and services agrees “We always joke around that in CMF the F stands for feelings so we joke that we design feelings The new Pixel comes in Sorta Seafoam and Kinda Coral while the Pixel Pro comes in Sorta Sunny and Cloudy White and both are available in Stormy Black Behind those five shades are years of work plenty of trial and error…and lots and lots of fine tuning “It s actually a very complex process Jenny says Mademore complex by COVID Both Jenny and Jess describe the color selection process as highly collaborative and hands on which was difficult to accomplish while working from home Designers aren t just working with their own teams but with those on the manufacturing and hardware side as well “We don t design color after the hardware design is done ーwe actually do it together Jenny says The Pixel and Pixel Pro s new premium look and feel influenced the direction of the new colors and the CMF team needed to see colors and touch items in order to select and eliminate the shades They don t only go hands on with the devices they do the same with sources of inspiration “I remember one time I really wanted to share this color because I thought it would be really appropriate for one of our products so I ended up sending my boss one of my sweaters through a courier delivery Jenny says “We found creative workarounds Photograph of various yellow to peach to pink items as well as a gold hoop earring and a Sorta Sunny Pixel Pro on a white table Photograph of the Stormy Black Pixel next to the black case and various items in different shades of black including a small shiny black spoon Photograph of the frosted white Pixel case next to various items of shades of white including a paper flow Photograph of the Kinda Coral Pixel next to a pink cube and a pale pink shell The team that designed the new Pixel and Pixel Pro case colors did as well “The CMF team would make models and then take photos of the models and I would try to go in and look at them in person and physically match the case combinations against the different phone colors says Nasreen Shad a Pixel Accessories product manager “Then we d render or photograph them and send them around to the team to review and see if what was and wasn t working In addition to the challenge of working remotely Nasreen s team was also working on something entirely new colorful translucent cases Nasreen says they didn t want to cover up the phones but complement them instead so they went with a translucent tinted plastic Each device has a case that corresponds to its color family but you can mix and match them for interesting new shades That process involved lots of experimenting For example what eventually became the Golden Glow case started out closer to a bronze color which didn t pair as well with the Stormy Black phone “We had to tune it to a peachy shade so that it looked good with its intended pairing Sorta Sunny but with everything else too That meant ordering more resins and color chips in different tones but it ended with some really beautiful effects Beautiful effects and tons of options “I posted a picture of all of the possible combinations you can make with the phones and the cases and people kept asking me how many phones did Google just release Nasreen laughs “And I had to be like No no no these are just the cases Google designers often only know the devices and colors by temporary internal code names It s up to their colleagues to come up with the names you see on the Google Store site now But one person who absolutely knows their official names is Lily Hackett a Product Marketing Manager who works on a team that names device colors “The way that we go about color naming is unique she says “We like to play on the color When you think about it it s actually very difficult to describe color and the colors we often use are subtle ーso we like to be specific with our approach to the name Because color can be so subjective one person s white and gold dress is another s black and blue dress Lily s team often checks in with CMF designers to make sure the words and names they re gravitating toward actually describe the colors accurately “It s so nice to go to color experts and say Is this right Is this a word you would use to describe this color Lily says their early brainstorming sessions can result in lists of or more options “It s truly a testament to our copywriting team When we were brainstorming for Stormy Black they had everything under the sun ーthey had everything under the moon It was incredible to see how many words they came up with These days everyone is looking ahead at new colors and new names but the team is excited to see the rest of the world finally get to see their work “I couldn t wait for them to come out Lily says “My favorite color was even the first to sell out on the Google Store I was like Yes everyone else loves it too	2021-10-29 16:45:00
Google	Official Google Blog	Email is 50 years old, and still where it’s @	http://feedproxy.google.com/~r/blogspot/MKuf/~3/pScRS07mrns/	Email is years old and still where it s years ago this month Ray Tomlinson sent the very first email He was a programmer working on ARPANET the system that laid the groundwork for what would become the internet as we know it today He tested the messaging system by sending emails to himself and later said that the first note was probably something like “QWERTYUIOP More than years after this breakthrough a Google engineer named Paul Buchheit conducted his own email experiments In a blog post Paul described the problem he was trying to solve “My email was a mess Important messages were hopelessly buried and conversations were a jumble…I couldn t always get to my email because it was stuck on one computer and web interfaces were unbearably clunky And I had spam A lot of it These pain points are part of what motivated Paul to come up with a better system ーGmail Buchheit created Gmail as a browser based email program that allowed users to easily search their own messages “With Gmail I got the opportunity to change email ーto build something that would work for me not against me He wasn t sure what the reception would be like but when he released a beta to fellow Googlers they wanted more Eventually Gmail launched to the public on April Its search function was lightning fast and it came with GB of storage ー times more than prevailing inboxes of the time But that wasn t enough to convince people it wasn t a joke The date ーApril Fool s Day ーlikely had something to do with it Despite the launch day hijinks Gmail won consumers over and became a central part of the work we do at Google But we never could have done it if Ray Tomlinson hadn t hit that sign and started it all years ago To celebrate we asked a few Googlers to share their favorite Gmail hacks Laura Mae Martin Executive Productivity Advisor“It s hard to answer old emails when there are shiny new ones coming in Use features like Snooze and Starred emails and different inbox setups to make it easier to stay on task John Shriver Blake Senior Product Manager Gmail Enterprise“I m a fan of confidential mode in Gmail It lets you protect sensitive information in messages and attachments and ensures that whoever receives the confidential email can t forward copy or print it Neena Kamath Product Lead Gmail What I love about Gmail is how it s evolved over the years Fifteen years ago I was obsessed with Conversation View Back then having all your emails about a single topic in one place hadn t been done before and it saved me so much time Now I m obsessed with Smart Reply It not only saves me time but also makes me more polite Bao Lam Head of Marketing Gmail amp Chat“Schedule send in Gmail means that I don t clutter up people s inboxes if I m catching up on emails at odd moments ーwhich is especially helpful when so many of us are working across different time zones You can learn more about the history of email Gmail and the power of the sign over on the Cloud blog	2021-10-29 16:08:00
python	Pythonタグが付けられた新着投稿 - Qiita	matplotlib と seaborn を日本語化する（Ubuntu）	https://qiita.com/Masahiro_T/items/33c3c88626b241912de5	環境matplotlibseabornubuntu対処法日本語フォントをインストールbashsudoaptinstallfontsipaexfontmatplotlibの設定ファイルを変更matplotlibの設定ファイルのパスを表示します。	2021-10-30 01:09:48
Program	[全てのタグ]の新着質問一覧｜teratail（テラテイル）	fgets()とatof()後に条件分岐がしたい	https://teratail.com/questions/366916?rss=all	fgetsとatof後に条件分岐がしたい回答ありがとうございます。	2021-10-30 01:52:29
Program	[全てのタグ]の新着質問一覧｜teratail（テラテイル）	HTMLフォームからスプレッドシートへの書き込み	https://teratail.com/questions/366915?rss=all	HTMLで作ったフォームからスプレッドシートへ書き込みをしたいです。	2021-10-30 01:40:14
Program	[全てのタグ]の新着質問一覧｜teratail（テラテイル）	ScriptableObjectを使ったインベントリでアイテムそれぞれに数字をテキスト表示したい。	https://teratail.com/questions/366914?rss=all	ScriptableObjectを使ったインベントリでアイテムそれぞれに数字をテキスト表示したい。	2021-10-30 01:28:36
Program	[全てのタグ]の新着質問一覧｜teratail（テラテイル）	PHP wordpree youtube 動画一覧をhome.php に取得する方法	https://teratail.com/questions/366913?rss=all	PHPwordpreeyoutube動画一覧をhomephpに取得する方法前提・実現したいことwordpreenbspyoutubenbsp動画一覧をhomephpnbspに取得する方法で問題を抱えています。	2021-10-30 01:28:19
Git	Gitタグが付けられた新着投稿 - Qiita	面倒なGitHubのPushを1秒で完了させる方法	https://qiita.com/harakazu_nanfg/items/b8e331556b556845435f	ブレイクタイム茶番そうして作業中のディレクトリに移動したら先ほど設定した「ap」を入力しよう※ディレクトリ内を何かしら編集することを忘れずに、、apこれでめでたくGitHubへのPushが秒でできるようになったこれがとても優秀なエンジニアである私がGitPushをするのが面倒だと思い過ぎたことで完成した「GitPushを秒でできる裏技」だしかしこの方法でPushした場合のコミットメッセージは「autopush」だけで非常に難があるため、これを業務でやって「なんなんだこの酷いコミットメッセージは」怒鳴られた時に「harakazunanfgがやれって言ってました」と言い訳しないようにだけ気をつけていただきたい。	2021-10-30 01:25:16
海外TECH	Ars Technica	Image in Facebook app may show Meta’s upcoming Apple Watch rival	https://arstechnica.com/?p=1808768	apple	2021-10-29 16:27:56
海外TECH	Ars Technica	Juno reveals deep 3D structure of Jupiter’s massive storms	https://arstechnica.com/?p=1808837	jupiter	2021-10-29 16:19:22
海外TECH	MakeUseOf	Are 3D Printed Utensils Food Grade? Here's How To Make Them Food Safe	https://www.makeuseof.com/are-3d-printed-utensils-food-safe/	Are D Printed Utensils Food Grade Here x s How To Make Them Food SafeIf you re going to D print utensils for use with food and drink you need to make sure it s non toxic Here s how	2021-10-29 16:45:21
海外TECH	MakeUseOf	Why You Shouldn't Always Close Apps on Your Android Phone	https://www.makeuseof.com/why-you-shouldnt-close-apps-android/	doing	2021-10-29 16:30:21
海外TECH	MakeUseOf	How to Add and Use Stickers on Discord	https://www.makeuseof.com/how-to-use-stickers-on-discord/	nitro	2021-10-29 16:16:21
海外TECH	DEV Community	40+ JavaScript Projects for Beginners – Easy Ideas to Get Started Coding JS [Beginners To Advance]	https://dev.to/jnb/40-javascript-projects-for-beginners-easy-ideas-to-get-started-coding-js-beginners-to-advance-54lh	JavaScript Projects for Beginners Easy Ideas to Get Started Coding JS Beginners To Advance The best way to learn a new programming language is to build projects I have created a list of beginner friendly project tutorials in Vanilla JavaScript React and TypeScript My advice for tutorials would be to watch the video build the project break it apart and rebuild it your own way Experiment with adding new features or using different methods That will test if you have really learned the concepts or not You can click on any of the projects listed below to jump to that section of the article Read More JavaScript Projects for Beginners Easy Ideas to Get Started Coding JS	2021-10-29 16:44:50
海外TECH	DEV Community	Being RESTful about your routes	https://dev.to/knowndecimal/being-restful-about-your-routes-1gmf	Being RESTful about your routesRuby on Rails advocates heavily for convention over configuration One of these core conventions is the baked in RESTful design for CRUD actions You re introduced to this convention when you re just starting out with Ruby on Rails Almost all web applications involve CRUD Create Read Update and Delete operations You may even find that the majority of the work your application does is CRUD Rails acknowledges this and provides many features to help simplify code doing CRUD The Rails docsThere is one big disadvantage about convention over configuration it s not always obvious what principle is applied why it s being applied and how it should be applied when things get harder A short introduction to RESTThe centerpiece of the REST principle is a resource Interacting with a resource should be done with stateless HTTP requests using one of the four verbs GET POST PUT PATCH or DELETE With stateless HTTP requests we mean that the server should be able to perform the operation with the given information from the client This information can be as part of the URI query string parameters body or headers In Rails a less pure implementation of REST is often used because most Rails apps are both the client and the server Uniform Resource Naming URN of resources is another part of REST The name of the resources should reflect what it is a singleton or a collection A list of repositories on Github is a collection resource that can be identified with the repositories URN An individual repository is a singleton resource and can be identified with the repositories URN Want to learn more about REST See Architectural Styles and the Design of Network based Software Architectures the dissertation of Roy Fielding REST on RailsIn Rails a resource is often translated to an ActiveRecord model However an ActiveRecord isn t the only resource in a Rails app A resource can be anything according to the REST design principles It can be an ActiveRecord model it can be a Plain Old Ruby Object PORO model it can be a service an operation and so on As we mentioned before Rails is opinionated Resourceful routing is one of these conventions In the Rails docs the preference for using resourceful routes is briefly mentioned While you should usually use resourceful routing Ruby on Rails provides a resourceful way to map the HTTP requests and URLs to controller actions There are multiple ways to define this mapping However the main two ways are to define a route resource and resources Let s take a look at a simple example that uses resources config routes rbRails application routes draw do resources repositories only i index show do resources collaborators only i index endendWith the routes above we re able to show a list of repositories and for each of these repositories we can show a list of collaborators Let s see how non restful routes can make sense at first but start to be a problem further down the road The non RESTful routeLet s say we want to invite a new collaborator to a repository by extending our previous example with non RESTful routes config routes rbRails application routes draw do resources repositories only i index show do resources collaborators only i index do post invite end endendSeems reasonable doesn t it We re inviting a new collaborator to a repository However now the new collaborator needs to be able to accept and decline the invite So we add it as a custom action to the controller too config routes rbRails application routes draw do resources repositories only i index show do resources collaborators only i index do post accept invite post decline invite post invite get show invite end endendThis approach still seems to be reasonable however our collaborators controller rb starts to grow It s not only responsible anymore for listing the collaborators but also for handling the invites It s not only breaking the Single Responsibility Principle SRP that goes hand in hand with a RESTful API design but it will also make it harder to maintain the codebase as it grows The collaborators controller rb will become longer harder to read and thus harder to maintain Let s see how the above example could be improved if one would follow a RESTful API design The RESTful routeIf we re following the REST design principles we will see that we re not actually inviting a new collaborator We re creating an invitation for the collaborator which they can accept or decline REST encourages us to think in terms of creating updating deleting resources rather than in actions So instead of inviting a collaborator we create an invitation for a collaborator The same is true regarding accepting or declining an invitation Wrapping your head around REST can take some time but it will make your code much cleaner and easier to maintain Let s take a look at a more RESTful approach to inviting new collaborators config routes rbRails application routes draw do resources repositories only i index show do resources collaborators only i index resources invitations only i show create update destroy endendNow we enable collaborators to see an invitation for a repository before accepting declining it We also allow them to accept the invitation by updating it and we allow them to decline the invitation by deleting it By expressing our intentions in a RESTful way we re separating the responsibilities of our controllers and are keeping them small This will not only make it easier to maintain and to read through them but it will also make them easier to test Following the REST design principles will help you to keep responsibilities of a controller to a minimum Whenever a new feature request comes in ask yourself how can I make this RESTful Splitting up your controllers following the REST design principles and reducing the responsibilities of a single controller will help you to make your applications more maintainable	2021-10-29 16:27:48
海外TECH	DEV Community	Data Engineering:Extract, Transform,and Load Using Talend Open Studio.	https://dev.to/wanjohichristopher/data-engineeringextract-transformand-load-using-talend-open-studio-4h0n	Data Engineering Extract Transform and Load Using Talend Open Studio Introduction To ETL Tools Hi there welcome back to the data engineering series I know most of you ask how do we do these so called ETL Or ELT processes as a Data Engineer I ve got you now don t worry Today we will be going through the process of Extracting Transforming and Loading Data step by step from SQL server to Postgres There are several ETL tools including Talend Pentaho Informatica Stitch Xplenty Alooma Panapoly Aws Glue etc I will introduce one ETL tool called Talend Introduction toTalend OpenStudioIt is important to use low code ETL tools In this article we will show how to install and get started with Talend We will use Talend later to Migrate Data from MSSQL Server to Postgres To get started we download Talend Open Studio for Linux here Note that you can also download for any Operating System in the same link After downloading you will receive multiple files run the Linux file which has a  sh file This is how it looks Launch Talend now This is how it should open It s now time to get our hands dirty Migrating Data from SQL Server to Postgres Using Talend ETLAt the Talend Studio we create a job design on your left hand side then create tBDInput for SQL Server and tBDOutput for Postgres These inputs and outputs DBs will be used during connections and migration At this point you can create a connection between the two databases Right Click on tBDInput Trigger the On subjob OK and drag it to tBDOutput to make the connection complete Ensure to enter correctly the database credentials i e username password and database name table name  which hosts the table and database to migrate alsoThis shows the SQL server credentials to enter After all the configuring and connecting both databases it s time now to start moving our data from SQL server to Postgres We write an SQL query to migrate the data To start with we start with SQL server by using the following command which gets data from the users table This is used for migration and logging using the get date function Now we now have to write an SQL query to insert data into Postgres In the UI we create a table and drop it if it exists then perform the action of insertion to custdatademo table as shown For Postgres configuration on Talend It s now time for us to run the job and migrate our data from MSSQL to Postgres Our job runs successfully Now it s time where we have been waiting for so long and it s time to see our output in Postgres Let s dive in fast and confirm our hard work The data has been MIgrated Successfully using Talend as you can see in PostgresAs you can see we have migrated the data successfully Thank you for being with us on this long journey This is a complete Extract Transform and Load ETL using Talend Happy learning guys Article Written by  WanjohiChristopher NicholasMusauRespects Neville Omwenga	2021-10-29 16:17:51
海外TECH	DEV Community	AWS - NLP newsletter October 2021	https://dev.to/aws/aws-nlp-newsletter-2021-oct-2e3o	AWS NLP newsletter October Hello world This is the monthly Natural Language Processing NLP newsletter covering everything related to NLP at AWS This is our third newsletter on Dev to If you missed our earlier episode here are Ep and Ep Feel free to leave comments share it on your social network to celebrate this new launch with us Service updates about NLP on AWSAmazon Lex launches progress updates for fulfillmentYou can now configure your Amazon Lex bots to provide periodic updates to users while their requests are processed Customer support conversations often require execution of business logic that can take some time to complete For example updating an itinerary on an airline reservation system may take a couple of minutes during peak hours Typically support agents put the call on hold and provide periodic updates e g “We are still processing your request thank you for your patience until the request is fulfilled Now you can easily configure your bot to automatically provide such periodic updates in a conversation With progress updates capability bot builders can quickly enhance the ability of virtual contact center agents and smart assistants New AWS Solution AWS QnABot a self service conversational chatbot built on Amazon LexThe AWS QnABot has now been released as an official AWS Solution Implementation The AWS QnABot is an open source multichannel multi language conversational chatbot built on Amazon Lex that responds to your customer s questions answers and feedback Without programming the AWS QnABot solution allows customers to quickly deploy self service conversational AI on multiple channels including their contact centers websites social media channels SMS text messaging or Amazon Alexa Amazon Transcribe now supports custom language models for streaming transcriptionAmazon Transcribe will now support custom language models CLM for streaming transcription Amazon Transcribe is an automatic speech recognition ASR service that makes it easy for you to add speech to text capabilities to your applications CLM allows you to leverage pre existing data to build a custom speech engine tailored for your transcription use case No prior machine learning experience required AWS ML Blog Transcribe Documentation Translate and analyze text using SQL functions with Amazon Redshift Amazon Translate and Amazon ComprehendYou already know how to use Amazon Redshift to transform data using simple SQL commands and built in functions Now you can also use Amazon Redshift to translate analyze and redact text fields thanks to Amazon Translate Amazon Comprehend and the power of Amazon Redshift supported AWS Lambda user defined functions UDFs Amazon Comprehend adds two Trusted Advisor checks Amazon Comprehend now supports two new AWS Trusted Advisor checks to help customers optimize the cost and security of Amazon Comprehend endpoints Today Amazon Comprehend checks are available in the AWS Business Support and AWS Enterprise Support plans The new checks are Underutilized endpoints Checks the throughput configuration of your endpoints and generates a warning when they are not actively used for any real time inference requests Endpoint permissions Checks the KMS key permissions for an endpoint whose underlying model was encrypted using customer managed keys If the customer managed key has been disabled or the key policy has been changed to alter the granted permissions for Amazon Comprehend for any reason the endpoint availability might be impacted Amazon Textract launches TIFF support and adds asynchronous support for receipts and invoices processingAmazon Textract now supports Tag Image File Format TIFF documents in addition to the PNG JPEG and PDF formats Customers can now process TIFF documents either synchronously or asynchronously using any of the following Amazon Textract APIs DetectDocumentText StartDocumentAnalysis StartDocumentTextDetection AnalyzeDocument and AnalyzeExpense Amazon Textract is a machine learning service that automatically extracts printed and handwritten text and data from any document NLP on SageMakerAmazon SageMaker JumpStart introduces new multimodal long form text tabular financial analysis toolsWith this new release you can use the new set of multimodal financial analysis tools within Amazon SageMaker JumpStart With these new tools you can enhance your tabular ML workflows with new insights from financial text documents and potentially help save up to weeks of development time Using the new SageMaker JumpStart Industry SDK you can easily retrieve common public financial documents including SEC filings and further process financial text documents with features such as summarization and scoring for sentiment litigiousness risk readability etc In addition you can access pre trained language models trained on financial text for transfer learning and use example notebooks for data retrieval text feature engineering multimodal classification and regression models AWS ML Blog AWS ML Blog AWS ML Blog JumpStart DocumentationOrganize product data to your taxonomy with Amazon SageMakerWhen companies deal with data that comes from various sources or the collection of this data has changed over time the data often becomes difficult to organize Perhaps you have product category names that are similar but don t match and on your website you want to surface these products as a group Therefore you need to go through the tedious work of manually creating a map from source to target to be able to transform the data into your own taxonomy In these cases we re not talking about a few hundred rows of data but more often many hundreds of thousands of rows with new data flowing in regularly In this post we discuss how to organize product data to your classification needs with Amazon SageMaker Bring structure to diverse documents with Amazon Textract and transformer based models on Amazon SageMakerFrom application forms to identity documents recent utility bills and bank statements many business processes today still rely on exchanging and analyzing human readable documentsーparticularly in industries like financial services and law In this post we show how you can use Amazon SageMaker an end to end platform for machine learning ML to automate especially challenging document analysis tasks with advanced ML models AWS Blog posts papers and moreCreate a dashboard with SEC text for financial NLP in Amazon SageMaker JumpStartIn this post the author showed how to curate a dataset of Securities Exchange Commission SEC filings use NLP for feature engineering on the dataset and present the features in a dashboard To get started you can refer to the example notebook in JumpStart titled Dashboarding SEC Filings You can also refer to the example notebook in JumpStart titled Create a TabText Dataset of SEC Filings in a Single API Call which contains more details of SEC forms retrieval summarization and NLP scoring Amazon Science Publication Sample selection guided by domain and task for cross domain targeted sentiment analysisBuilding supervised targeted sentiment analysis models for a new target domain requires substantial annotation effort since most datasets for this task are domain specific Domain adaptation for this task has two dimensions the nature of targets and the opinion words used to describe sentiment towards the target We present a data sampling strategy informed by domain differences across these two dimensions with the goal of selecting a small number of examples thereby minimizing annotation effort This obtains performance in the range compared to the full supervised model using only of the full training data YouTube demo video Amazon Transcribe video snacks Using vocabulary filters Amazon Transcribe is a automatic speech recognition service that can be used when you have audio and video that contains speech you want to convert to text You can mask remove or tag words you don t want in your transcription results with vocabulary filtering For example you can use vocabulary filtering to prevent the display of offensive or profane terms In the demo we will customize Transcribe to mask swear words that we recently encountered in a famous play written by William Shakespeare ways conversational AI and Amazon Lex help the public sector transform customer engagementConversational artificial intelligence AI and chatbots can be used to transform the customer experience enhance engagement improve services and help scale more simply Learn how conversational AI and chatbots help public sector organizations Community contentWorkshop Getting started with Amazon Sagemaker Train a Hugging Face Transformers and deploy itLearn how to use Amazon SageMaker to train a Hugging Face Transformer model and deploy it afterward Prepare and upload a test dataset to S prepare a fine tuning script to be used with Amazon SageMaker Training jobs Launch a training job and store the trained model into S and Deploy the model after successful training GitHub RepositoryOctober “HuggingFace Blog entries Showcase Your Projects in Spaces using Gradio Hosting your Models and Datasets on Hugging Face Spaces using StreamlitFine tuning CLIP with Remote Sensing Satellite images and captionsThe Age of Machine Learning As Code Has ArrivedTrain a Sentence Embedding Model with B Training PairsLarge Language Models A New Moore s Law Course Launch Community Event Upcoming NLP eventsBoth community events and AWS eventsGoing Production Deploying Scaling amp Monitoring Hugging Face Transformer models Hugging FaceTuesday November nd PM to PM CESTPie amp AI Suisse Trustworthiness of AI models Improving NLP with Causality MeetupWednesday November PM to PM CESTNLP inference optimization on Amazon SageMaker in NDR conferenceTuesday November AM to PM CESTAnalysing Politeness Can NLP Tools Help MeetupWednesday November PM to PM CEST Stay in touch with NLP on AWSOur contact aws nlp amazon com Email us about your awesome project about NLP on AWS let us know which post in the newsletter helped your NLP journey other things that you want us to post on the newsletter Talk to you soon	2021-10-29 16:11:23
海外TECH	DEV Community	Hotwire with StimulusReflex	https://dev.to/julianrubisch/hotwire-with-stimulusreflex-2bh5	enjoy	2021-10-29 16:09:36
Apple	AppleInsider - Frontpage News	Apple's 140W 16-inch MacBook Pro charger gets the teardown treatment	https://appleinsider.com/articles/21/10/29/chargerlab-tears-down-apples-140w-macbook-pro-charger?utm_medium=rss	Apple x s W inch MacBook Pro charger gets the teardown treatmentPeripherals company ChargerLab has taken apart Apple s first ever GaN charger the W one for the inch MacBook Pro and detailed every component sourced Do not do this at homeIt was already known that Apple uses gallium nitride in its new W charger for the inch and that it was done to improve power delivery while keeping a small footprint Now a teardown company has stripped a unit to uncover every detail of its construction and design Read more	2021-10-29 16:48:06
海外TECH	Engadget	Boston Dynamics wants you to know its Spot robot has moves like Jagger	https://www.engadget.com/boston-dynamics-spot-robot-start-me-up-164722012.html?src=rss	Boston Dynamics wants you to know its Spot robot has moves like JaggerThe last time we saw Spot dance it was with the entire Boston Dynamics family at the end of last year That was mostly a showcase of how much Atlas the company s bipedal robot had come from the days when it could barely walk In a new solo display we get to see it move to “Start Me Up from The Rolling Stones in honor of the th anniversary of their album Tattoo You And if you thought Spot dancing was too close to the uncanny valley wait until you see it lip sync It is truly the stuff of nightmare fuel Boston Dynamics releases these types of videos at a regular cadence almost as if to remind those watching “Yes we still exist Earlier in the year Hyundai completed its acquisition of the company making it the third owner of the firm in the span of a decade More recently the automaker detailed how it would put Spot to work at its assembly plants in South Korea Bet Spot isn t doing too much dancing there nbsp nbsp	2021-10-29 16:47:22
海外TECH	Engadget	How Resident Evil's tallest and most terrifying vampire lady, Alcina Dimitrescu, came to life	https://www.engadget.com/resident-evil-village-lady-dimitrescu-mocap-interview-halloween-2021-163032866.html?src=rss	How Resident Evil x s tallest and most terrifying vampire lady Alcina Dimitrescu came to lifeCountess Alcina Dimitrescu isn t technically a vampire but she s definitely an icon From the moment she appeared in an early trailer for Resident Evil Village Lady Dimitrescu captured the attention of the internet her towering sultry frame her sickly sweet smile her massive hat her proper yet authoritative tone Viewers were instantly obsessed labeling her quot tall vampire lady quot and demanding to know everything about her which prompted Capcom to reveal her exact height ー quot including her hat and heels ーmonths before Village launched in May In game Lady Dimitrescu commands three fly infested murderous daughters and hunts the main character Ethan Winters across the opulent rooms of her castle She crouches through doorways and slices at Ethan with long blade like fingernails hurling insults like quot rat quot and quot man thing quot at him the entire time a mutant dominatrix in a flowing white dress Twilight SparkleThe intrigue surrounding Lady Dimitrescu has persisted since Village s launch and fans are betting or maybe just hoping really hard that the game s first bit of DLC will focus on her specifically Until then and in the spirit of Spooky Season we have insight into Lady Dimitrescu s creation from Village presentation director Masato Miyazaki the person in charge of the game s motion capture process Earlier this year Miyazaki shared details with Engadget about how Lady Dimitrescu came to be from concept to mocap including the ways actress Maggie Robertson brought her to legendary life Engadget Did Lady Dimitrescu s design change throughout development Masato Miyazaki Alcina Dimitrescu s incredible height was conceived from the beginning and was not changed during development The same goes for her wide brim hat and her white dress as well However the characteristic of her long protruding nails was something added part way through the development process It was an idea that was implemented later as a means of adding physical elements that would make her more terrifying when you encounter her Lady Dimitrescu is alluring and seductive ー was she always meant to be a sultry character or did that emerge during mocap In the early stages of development she was described as a bewitching character who would capture and toy with her victims She was designed to embody equal parts beauty and horror Based on this the scenario writer fleshed her characterization out even further with dialogue but she wasn t fully realized just yet It was through Maggie s performance that the character was finally given life As with any character I believe that the moment the script is handed over to the actor the character becomes theirs The character s personality and intentions are very much refined by the actor The character Lady Dimitrescu was truly realized and came to fruition with each of Maggie s performances CapcomWhat tricks did you use to make Maggie Robertson as tall as possible during mocap sessions Although Maggie Robertson is quite tall herself she still cannot reach the height that we envisioned for Lady Dimitrescu Utilizing some means of extending out her height would jeopardize her performance so it wasn t something we could consider We asked Maggie to act naturally However that still left us with the height difference between Maggie and Lady Dimitrescu that had to be addressed We devised a few methods to counteract this issue First off we shot with a mixture of backgrounds according to two standards human scale and Lady Dimitrescu scale While other characters performed with surroundings that fit human scale Lady Dimitrescu s acting was done in front of a background that fit her scale Everyone performed with one another but with this mixture of environments We figured this would be the best means of allowing the actors to give their best performances without any kind of impediment We made sure the furniture was laid out in a way so that the actors would be facing each other The other aspect that we made sure to stay conscious of was making sure the actors lines of sight were in the right positions We set up markers so that the actors could imagine the correct height These markers show the correct position of the eye lines and where the limbs actually are It s a simple adjustment but it makes a big difference in the actors performances How the sausage gets made part This is what we used to mark Lady D s actual size and eye lines Clearly she s grumpy after chasing people around the castle all day thegrumpystandin LadyDimitrescu ResidentEvilVillage check out that hat tho It s a look pic twitter com IgfKLーMaggie Robertson maggiethebard May The third adjustment was the rig itself We carefully set up the rigs of the CG characters to gracefully handle as much of the physical differences between the character and actor as possible We wanted to make sure that we set things up so that the animators would have a relatively easier time handling any kind of miscalculations that happened along the way What was the strangest prop you used throughout the mocap process There are several but there are two that I would like to mention The first is the cane carried by the old woman we meet at the beginning of the village When you see it in the game it s a strange artifact with a lot of components jingling about The studio crew crafted a prop that resembles the design The second is the goggles that our actor Todd wore as he played the role of Ethan Winters Since the game is from a first person perspective through the eyes of Ethan Winters the camera movements are based on the movements of Todd s head Todd reprises his role as Ethan from the previous game and his performances are absolutely excellent but there were moments where the camera would go wild during more heated scenarios In order to suppress this from happening the studio engineer created a pair of special goggles by hand nbsp They actually turned out to look very similar to swimming goggles The assumption was that the narrower field of vision would result in less head movement I m still not entirely sure how effective they ended up being but the engineer s enthusiasm and Todd having fun wearing the goggles are one of the many wonderful memories I have from the whole experience	2021-10-29 16:30:32
海外TECH	Engadget	We won't have electric airplanes until battery tech improves	https://www.engadget.com/we-wont-have-electric-airplanes-until-battery-tech-improves-160058357.html?src=rss	We won x t have electric airplanes until battery tech improvesToday s commercial airliners are not exactly fuel efficient The average for example burns through a gallon of kerosene based fuel every second that it flies And with billion people expected to take to the skies annually by carbon free alternatives to Jet A will be necessary in order to offset the industry s impact on global warming We are nearing the age of electric airplanes Pioneering researchers scientists and entrepreneurs have been working on the dream of electrified flight since the latter part of the th century when heavy lead acid batteries were loaded onto early airships to power their propellers We ve also seen a number of ahem novel means of powering aircraft while in flight throughout the years from conductive tethers stretching back down to the ground to solar panels to microwave energy transmission but it wasn t until the advent of relatively more power dense Nickel cadmium NiCad battery technology that human scale free flying electric planes became technically feasible But even as battery chemistries have evolved and energy densities have risen over the past few decades today s state of the art Lithium ion cells pose the same quandry to the aviation industry as they do to the automotive how to properly balance the energy to weight ratio of their batteries “If a jumbo jet were to use today s batteries million pounds of batteries would be required just to generate the power of the jet engine it would be replacing University of Houston Energy Fellow Emily Pickrell opined in Forbes earlier this year “This weight would effectively need an additional eight jet planes just to carry that weight quot And as Li ion technology has fully matured further increases to its energy density have fallen to below five percent with each annual iteration which is why a number of researchers and battery companies are already looking for the next breakthrough battery chemistry ーwhether that s Sodium ion Na ion Lithium metal Li metal Lithium Sulphur Li S or Zinc air Zn air Regardless of composition batteries need to get a whole lot lighter and more energy dense if they re going to attack and dethrone jet fuel which with an energy density of kWh L makes the flammable liquid about times as energy dense as today s best li ions To be fair though due to inefficiencies inherent to internal combustion engines that figure drops to around times the energy density of a li ion battery if you re comparing equal weights of fuel and batteries For example a Tesla Model s li ion based battery boasts an energy density of Wh kg while CATL announced earlier this year that it had built a sodium ion battery with Wh kg density though it hopes to get that up to Wh kg by Lithium sulfur batteries have shown the capacity to hold up to Wh kg though that technology faces significant longevity hurdles i e the chemistry tends to eat through electrodes before they can be widely used Currently and person small aircraft outfitted with electric power systems typically operate at Wh kg of specific energy but industry experts expect energy densities will have to hit Wh kg before the electric aviation industry really takes off ーsomething that could happen within the next few years according to Tesla CEO Elon Musk Wh kg with high cycle life produced in volume not just a lab is not far Probably to years ーElon Musk elonmusk August Preventing and mitigating thermal runaway is another critical test for electric aviation When a battery cell or even an area within a single cell malfunctions due to mechanical thermal or electrochemical failure its temperature can rise beyond safe levels causing the cell to first produce lithium off gasses causing the cell walls to bulge then rupture releasing the entirety of its energy reserve When a cell bursts it can damage and overheat surrounding cells setting off a cascading failure that results in explosion and fire When that happens to a Chevy Volt the car will likely be a write off fingers crossed it didn t also set your house on fire but if such a failure were to occur in flight on an electrified the loss of life would be catastrophic To minimize the chances of a full blown runaway from occurring early detection of cell failures is key As off gassing typically occurs minutes before a cell ruptures the presence of a monitoring system which compares sensors positioned close to a li ion battery against those collected by a reference sensor further away can alert for the presence of a failing cell And to negate any gases that have already been released fire suppression systems armed with inert gas ーto prevent the offgasses from reaching combustible levels when mixed with atmospheric oxygen ーcan be employed as well Of course regular maintenance and robust inspections also help prevent cell failures before the situation becomes explosive Rolls RoyceBattery electric planes will also provide unique challenges in balancing air speed and range though for Rolls Royce it s not even a question ーspeed all the way Over the past few years Rolls Royce has been quietly working on Project ACCEL accelerating electrification of flight building a battery powered racing plane dubbed Spirit of Innovation in an effort to set a new world air speed record The record was previously set in when an electric powered Extra LE using a Siemens eAircraft built power plant notched a mph kph top speed over a kilometer long course The feat was certified by the World Air Sports Federation FAI as the fastest electrically powered flight by an aircraft weighing less than kg at takeoff beating the previous record set in by just over mph kph In addition to the kilometer record Rolls Royce has the opportunity to also set FAI records for a km distance and “time to altitude basically how quickly the plane can take off and reach a specific height “It needs to be a significant number Rolls Royce Director of Engineering and Technology Civil Aerospace Simon Burr told Aerosociety “We re planning to fly over mph We ll see how high we can get to Rolls RoyceFor its attempt Rolls Royce ーwhich is partnering with the UK s YASA electric motor manufacturer and start up Electroflight which makes bespoke battery systems ーhas acquired a pair of Sharp Nemesis NXT twin seat air racers One has been used for ground testing while the second will conduct the actual flights The Nemesis NXT already holds the km FAI record with a recorded top speed of mph km using a hp Lycoming internal combustion engine The Rolls Royce team has swapped that Lycoming engine out for a trio of YASA v electric motors producing around kW hp while the fuel tank has been replaced with three independent battery packs Rolls Royce“The main challenge of electrification is weight Rolls Royce Flight Test Engineer Andy Roberts said during a September media briefing Not only did the cell battery system aboard the Nemesis NXT shift the aircraft s center of balance the kg battery system also doesn t get lighter over time as conventional fuel tanks would which could impact the plane s performance during the later stages of the run The batteries are so substantial that Rolls Royce Chief Test Pilot Phill O Dell had to lose kg of bodyweight to help keep the overall aircraft weight within operating margins Thermal runaway is a very real concern for the Rolls Royce team as they ll be pushing these batteries to their absolute limits during the flight In order to mitigate this issue cells are separated by liquid cooling plates and stored in cork wrapped fireproof cases the porous cork material helps diffuse heat Should a cell overheat to the point of venting off gasses the plane is equipped with an inert gas suppression and ventilation system as well On September th the Spirit of Innovation made its maiden test flight from the UK Ministry of Defence s Boscombe Down airfield flying for minutes The company hopes to have the Nemesis ready for an official run at the record before the end of this year “The first flight of the Spirit of Innovation is a great achievement We are focused on producing the technology breakthroughs society needs to decarbonize transport across air land and sea and capture the economic opportunity of the transition to net zero Warren East Rolls Royce CEO said in a statement “This is not only about breaking a world record the advanced battery and propulsion technology developed for this programme has exciting applications for the Urban Air Mobility market Rolls Royce is far from the only company pursuing electric aircraft technology no matter how much faster it is than the competition From tiny startups to industry stalwarts ーeven NASA ーcompanies and governments around the world are racing to develop commercially viable electric aircraft both for passenger flights and cargo hauling nbsp Guglielmo Mangiapane reutersBye Aerospace for example builds electrified seat trainer planes called the eFlyer similar in function to Diamond Aircraft s eAircraft Slovenian aircraft manufacturer Pipistrel has been selling its Alpha Electro the first electric plane to earn FAA certification since On the other end of the spectrum you have aerospace giants like Airbus developing the Air Race E which the company claims is the world s first all electric air race series when it starts up later this year better get with the times Red Bull Air Race and demonstrators like the City Airbus a seat eVTOL These electric vertical take off and landing capable vehicles have become a popular option for fossil fuel free air travel such as Cadillac s single seater concept the build it yourself Jetson Aero China s EHang AAV Uber s since abandoned air taxi scheme or Volocopter s ongoing air taxi scheme Unfortunately despite all the research into and hype surrounding electrified air travel many industry experts remain skeptical that we ll see its widespread adoption for at least a few more decades ーat least for large scale airframes like the Boeing or Airbus A Until battery technologies become sufficiently robust we ll most likely see eVTOLS restricted to short hop intracity duties for the foreseeable future eventually expanding out to inter city jaunts and regional commuter jets Still it beats sitting in traffic	2021-10-29 16:00:58
Cisco	Cisco Blog	Mining’s Digital Imperative	https://blogs.cisco.com/energy/mining-digital-imperative	Mining s Digital ImperativeThe cyclical boom and bust of mining is a familiar rhythm to industry veterans Leveraging the opportunities that come from this cycle requires mines to be adaptable and demands a clear view of the operational realities The digital strategies available today provide a viable solution to these requirements They offer a clear and current view	2021-10-29 16:36:34
海外科学	NYT > Science	How to Decide Which Covid Booster Shot to Get	https://www.nytimes.com/2021/10/27/well/live/covid-booster-shot-mix-and-match.html	original	2021-10-29 16:13:35
海外科学	BBC News - Science & Environment	Climate change: What are the big polluters doing to cut carbon emissions?	https://www.bbc.co.uk/news/58956714?at_medium=RSS&at_campaign=KARANGA	emissions	2021-10-29 16:31:31
金融	金融庁ホームページ	鈴木財務大臣兼内閣府特命担当大臣閣議後記者会見の概要（令和３年10月26日）を公表しました。	https://www.fsa.go.jp/common/conference/minister/2021b/20211026-1.html	内閣府特命担当大臣	2021-10-29 17:22:00
金融	金融庁ホームページ	鈴木財務大臣兼内閣府特命担当大臣閣議後記者会見の概要（令和３年10月12日）を公表しました。	https://www.fsa.go.jp/common/conference/minister/2021b/20211012-1.html	内閣府特命担当大臣	2021-10-29 17:20:00
金融	金融庁ホームページ	金融機関における貸付条件の変更等の状況について更新しました。	https://www.fsa.go.jp/ordinary/coronavirus202001/kashitsuke/20200430.html	金融機関	2021-10-29 17:00:00
金融	金融庁ホームページ	「新型コロナウイルス感染症関連情報」特設ページを更新しました。	https://www.fsa.go.jp/ordinary/coronavirus202001/press.html	新型コロナウイルス	2021-10-29 17:00:00
金融	金融庁ホームページ	貸金業関係資料集を更新しました。	https://www.fsa.go.jp/status/kasikin/20211029/index.html	関係	2021-10-29 17:00:00
金融	金融庁ホームページ	レバレッジ比率規制に係る告示の一部改正（案）等について公表しました。	https://www.fsa.go.jp/news/r3/ginkou/20211029-2.html	規制	2021-10-29 17:00:00
金融	金融庁ホームページ	バーゼル銀行監督委員会、BIS決済・市場インフラ委員会及び証券監督者国際機構による市中協議文書「証拠金の慣行に関するレビュー」について掲載しました。	https://www.fsa.go.jp/inter/ios/20211029.html	証券監督者国際機構	2021-10-29 17:00:00
金融	金融庁ホームページ	「令和３年度金融庁政策評価実施計画」等の策定について公表しました。	https://www.fsa.go.jp/news/r3/sonota/20211029_2/20211029_2.html	政策評価	2021-10-29 16:59:00
ニュース	BBC News - Home	Fishing row: UK warns France it could retaliate over threats	https://www.bbc.co.uk/news/uk-59087420?at_medium=RSS&at_campaign=KARANGA	escalates	2021-10-29 16:50:42
ニュース	BBC News - Home	Penelope Jackson guilty of murdering husband over dinner row	https://www.bbc.co.uk/news/uk-england-somerset-59096177?at_medium=RSS&at_campaign=KARANGA	somerset	2021-10-29 16:47:06
ニュース	BBC News - Home	Queen advised to rest for two more weeks	https://www.bbc.co.uk/news/uk-59097106?at_medium=RSS&at_campaign=KARANGA	buckingham	2021-10-29 16:50:13
ニュース	BBC News - Home	Menopause: HRT prescription cost cut welcomed by campaigners	https://www.bbc.co.uk/news/uk-politics-59088557?at_medium=RSS&at_campaign=KARANGA	england	2021-10-29 16:14:55
ニュース	BBC News - Home	Greta Thunberg: Activist calls on banks to stop funding climate 'destruction'	https://www.bbc.co.uk/news/uk-politics-59090943?at_medium=RSS&at_campaign=KARANGA	activist	2021-10-29 16:11:15
ニュース	BBC News - Home	Covid-19 in the UK: How many coronavirus cases are there in my area?	https://www.bbc.co.uk/news/uk-51768274?at_medium=RSS&at_campaign=KARANGA	cases	2021-10-29 16:20:09
ニュース	BBC News - Home	Climate change: What are the big polluters doing to cut carbon emissions?	https://www.bbc.co.uk/news/58956714?at_medium=RSS&at_campaign=KARANGA	emissions	2021-10-29 16:31:31
Azure	Azure の更新情報	Logic Apps Standard Plan updates in public preview	https://azure.microsoft.com/ja-jp/updates/logic-apps-standard-plan-updates-2/	Logic Apps Standard Plan updates in public previewLogic Apps Standard was released at Ignite and since then we have added a lot of features to enable better enterprise integration between mission critical systems on cloud on premises and in a hybrid way with runtime available on Azure Azure Arc and locally This post includes capabilities now available in public preview	2021-10-29 16:25:21
Azure	Azure の更新情報	Querying Delta Lake files using T-SQL in Azure Synapse Analytics is generally available	https://azure.microsoft.com/ja-jp/updates/querying-delta-lake-files-using-tsql-in-azure-synapse-analytics-is-generally-available/	azure	2021-10-29 16:24:50
GCP	Cloud Blog	Django ORM support for Cloud Spanner is now Generally Available	https://cloud.google.com/blog/topics/developers-practitioners/django-orm-support-cloud-spanner-now-generally-available/	Django ORM support for Cloud Spanner is now Generally AvailableToday we re happy to announce GA support for Google Cloud Spanner in the Django ORM The django google spanner package is a third party database backend for Cloud Spanner powered by the Cloud Spanner Python client library The Django ORM is a powerful standalone component of the Django web framework that maps Python objects to relational data It provides a nice Pythonic interface to the underlying database and includes tools for automatically generating schema changes and managing schema version history With this integration Django applications can now take advantage of Cloud Spanner s high availability and consistency at scale We ll follow the Django tutorial below to create a new project and start writing data to Cloud Spanner This is a follow up to the Introducing Django ORM support for Cloud Spanner blog post which we published during the Beta launch We have updated the tutorial to work with the Django library If you re already using Django with another database backend you can skip down to Migrating an existing Django project to Cloud Spanner for instructions on switching to Cloud Spanner You can also read the documentation here and follow the repository here  Changes since the Beta releaseThe library supports Django version x and x Both versions are long term support LTS releases for the Django project The minimum required Python version is NUMERIC data type support is now available We have also added support for JSON object storage and retrieval with Django x support but querying inside the JSONfield is not supported in the current django google spanner release This feature is being worked on and can be tracked here  Support for PositiveBigIntegerField PositiveIntegerField and PositiveSmallIntegerField were added along with relevant check constraints InstallationTo use django google spanner you ll need a working Python installation and Django project The library requires Django or Django and Python gt If you re new to Django see the Django getting started guide specific to the Django version you are using For the tutorial below we will be using Django but the process is similar for Django as well You ll also need an active Google Cloud project with the Cloud Spanner API enabled For more details on getting started with Cloud Spanner see the Cloud Spanner getting started guide Django applications are typically configured to use a single database If you re an existing Cloud Spanner customer you should already have a database suitable for use with your Django application If you don t already have a Cloud Spanner database or want to start from scratch for a new Django application you can create a new instance and database using the Google Cloud SDK To install the Cloud Spanner database backend package Next start a new Django project django google spanner provides a Django application named django spanner To use the Cloud Spanner database backend this application needs to be the first entry in INSTALLED APPS in your application s settings py file The django spanner application changes the default behavior of Django s AutoField so that it generates random instead of automatically incrementing sequential values We do this to avoid a common anti pattern in Cloud Spanner usage Configure the database engine by setting the project instance and database name To run your code locally during development and testing you ll need to authenticate with Application Default Credentials or set the GOOGLE APPLICATION CREDENTIALS environment variable to authenticate using a service account This library delegates authentication to the Cloud Spanner Python client library If you re already using this or another client library successfully you shouldn t have to do anything new to authenticate from your Django application For more information see the client libraries documentation on setting up authentication  Working with django google spannerUnder the hood django google spanner uses the Cloud Spanner Python client library which communicates with Cloud Spanner via its gRPC API The client library also manages Cloud Spanner session lifetimes and provides sane request timeout and retry defaults To support the Django ORM we added an implementation of the Python Database API Specification or DB API to the client library in the google cloud spanner dbapi package This package handles Cloud Spanner database connections provides a standard cursor for iterating over streaming results and seamlessly retries queries and DML statements in aborted transactions In the future we hope to use this package to support other libraries and ORMs that are compatible with the DB API including SQLAlchemy Django ships with a powerful schema version control system known as migrations Each migration describes a change to a Django model that results in a schema change Django tracks migrations in an internal django migrations table and includes tools for migrating data between schema versions and generating migrations automatically from an app s models django google spanner provides backend support for Cloud Spanner by converting Django migrations into DDL statements namely CREATE TABLE and ALTER TABLE  to be run at migration time Following the Django tutorial let s see how the client library interacts with the Cloud Spanner API The example that follows starts from the Database setup step of Tutorial and assumes you ve already created the mysite and polls apps from the first part of the tutorial After configuring database backend as described above we can run the initial migrations for the project After running the migrations we can see the tables and indexes Django created in the GCP Cloud Console Alternatively we can inspect information schema tables to display the tables Django created using the Google Cloud SDK Note that this will display Spanner internal tables too including SPANNER SYS and INFORMATION SCHEMA tables These are omitted in the example above Check the table schema of any table in GCP cloud console by clicking the SHOW EQUIVALENT DDL link on the table detail page in the Cloud Console Now following the Playing with the API section of the tutorial let s create and modify some objects in the polls app and see how the changes are persisted in Cloud Spanner In the example below each code segment from the tutorial is followed by any SQL statements executed by Django and a partial list of resulting Cloud Spanner API requests including their arguments To see the generated SQL statements yourself you can enable the django db backends logger Query the empty Questions tableThis code results in the SQL statement We have skipped the internal Cloud Spanner API calls that are made those details can be found in our earlier blog post Create and save a new Question objectThis code results in the SQL statement Modify an existing QuestionThis code results in the SQL statement Migrating an existing Django project to Cloud SpannerTo migrate a Django project from another database to Cloud Spanner we can use Django s built in support for multiple database connections This feature allows us to connect to two databases at once to read from one and write to another Suppose you want to move your application s data from SQLite to Cloud Spanner Assuming the existing database connection is already configured as default we can add a second database connection to Cloud Spanner We ll call this connection spanner As in the tutorial running python manage py migrate will create tables and indexes for all models in the project By default migrate will run on all configured database connections and generate DDL specific to each database backend After running migrate both databases should have equivalent schemas but the new Cloud Spanner database will still be empty Since Django automatically generates the schema from the project s models it s a good idea to check that the generated DDL follows Cloud Spanner best practices You can adjust the project s models accordingly in a separate migration after copying data into Cloud Spanner There are several options for copying data into Cloud Spanner including using HarbourBridge to import data from a PostgreSQL or MySQL database or Dataflow to import Avro files Any option will work as long as the imported data matches the new schema but the easiest if not the fastest way to copy data between databases is by using Django itself Consider the models we created in the tutorial In this code snippet we read all Questions and Choices from the SQLite database and then write them to Cloud Spanner For each row in each table in the existing database we Read the row and store it in memory as a Django model objectUnset the primary key andWrite it back to the new database at which point it gets assigned a new primary key Note that we need to update foreign keys to use the newly generated primary keys too Also note that we call question choice set all  before we change question s primary key otherwise the QuerySet would be evaluated lazily using the wrong key This is a naive example meant to be easy to understand but not necessarily fast It makes a separate SELECT …FROM polls choice query for each Question Since we know ahead of time that we re going to read all Choices in the database we can reduce this to a single query with Choice objects all select related question In general it should be possible to write your migration logic in a way that takes advantage of your project s schema e g by using bulk update instead of a separate request to write each row This logic can take the form of a code snippet to be run in the Django shell as above a separate script or a Django data migration After migrating from the old database to Cloud Spanner you can remove the configuration for the old database connection and rename the Cloud Spanner connection to default LimitationsNote that some Django database features are disabled because they re not compatible with the Cloud Spanner API As Django ships with a comprehensive test suite you can look at the list of Django tests that we skip for a detailed list of Django features that aren t yet supported by python spanner django Customers using the Cloud Spanner Emulator may see different behavior than the Cloud Spanner service for instance because the emulator doesn t support concurrent transactions See the Cloud Spanner Emulator documentation for a list of limitations and differences from the Cloud Spanner service We recommend that you go through the list of additional limitations of Spanner and django google spanner both before deploying any projects using this library These limitations are documented here  Getting involvedWe d love to hear from you especially if you re using the Cloud Spanner Python client library with a Django application now or if you re an existing Cloud Spanner customer who is considering using Django for new projects The project is open source and you can comment report bugs and open pull requests on Github See alsoDjango Cloud Spanner Python client library documentationCloud Spanner Python client library documentationCloud Spanner product documentationDjango documentationDjango tutorialRelated ArticleCloud Spanner connectivity using JetBrains IDEsYou can now browse the database schema and query data stored in Cloud Spanner directly from your JetBrains IDE In this post I will show Read Article	2021-10-29 16:15:00