投稿時間:2022-12-12 18:20:12 RSSフィード2022-12-12 18:00 分まとめ(26件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
IT	ITmedia 総合記事一覧	[ITmedia News] Meta、アバターストアを日本でもスタート　プラダなどのバーチャル服を販売	https://www.itmedia.co.jp/news/articles/2212/12/news152.html	facebook	2022-12-12 17:30:00
python	Pythonタグが付けられた新着投稿 - Qiita	【Python】.pydファイルを配置する	https://qiita.com/Jobs417/items/27646e6aaf170804b761	cpython	2022-12-12 17:34:49
python	Pythonタグが付けられた新着投稿 - Qiita	Xmasまでに絶対に彼女を作りたい！	https://qiita.com/Kentea/items/790095b6ad58e18334a0	adventcalendar	2022-12-12 17:25:51
python	Pythonタグが付けられた新着投稿 - Qiita	python – 複数回のユーザー入力 (input) を標準入力としてコマンド実行する	https://qiita.com/YumaInaura/items/9c625c9de713757bf833	echonnpythonexample	2022-12-12 17:03:58
js	JavaScriptタグが付けられた新着投稿 - Qiita	ChatGPTさんに企画候補・企画書の生成・コーディングまでやってもらった (小説風)	https://qiita.com/alclimb/items/91d9183aed12fd6db4f8	chatgpt	2022-12-12 17:50:47
js	JavaScriptタグが付けられた新着投稿 - Qiita	dynamic importでのページ分割＋CSSファイル分割したら読み込みパフォーマンスが３倍改善した話	https://qiita.com/shota_oha/items/a13b4dcad9f430dd1401	dynamicimport	2022-12-12 17:41:24
技術ブログ	KAYAC engineers' blog	全国統一Webエンジニア世論調査	https://techblog.kayac.com/yoronChosa2022	docsgooglecomこれを機に、エンジニアと非エンジニアの仕事が円滑に進むことを願っています。	2022-12-12 18:00:00
海外TECH	DEV Community	AWS open source newsletter, #138	https://dev.to/aws/aws-open-source-newsletter-138-461a	AWS open source newsletter December th Instalment WelcomeWelcome to the AWS open source newsletter edition After a week off due to re Invent this edition is packed with content on many of the open source related announcements As always we have a great line up of new projects for you to practice your four freedoms on In no particular order we have projects like eks node viewer a nice visualisation tool for your Amazon EKS clusters pg tle a great new project to make your PostgreSQL environments safer dyna a fun project that finally turns Amazon Route into a database dynamodb mass migrations a tool to help you migrate to Amazon DynamoDB visual asset management system a very nice digital asset management tool fast differential privacy implement differential privacy in your PyTorch models migration hadoop to emr tco simulator a handy total cost of ownership calculator for Amazon EMR realtime toxicity detection a tool to help you stay on top of your online communities functionclarity a very cool tool to check the integrity of your serverless functions before executing and many more We also feature this week content on a broad array of open source technologies such as Rez Terraform PostgreSQL Apache Hudi Apache Iceberg Delta Lake Amazon EMR Apache Iceberg Apache Spark OpenZFS Ray MySQL Kubernetes Apache Kafka Open Invention Network AWS IoT Greengrass Ray Modin Amazon Corretto Firecracker DeeQu AWS SDK for pandas Amazon Braket Yocto Logshell MQTT Redis and many more Finally make sure you check out the Video section where I share what i think are the best videos from re Invent on open source Amazon Joins the Open Invention NetworkOpen Invention Network OIN is a company that acquires patents and licenses them royalty free to its community members who in turn agree not to assert their own patents against Linux and Linux related systems and applications Announced last week David Nalley wrote about Amazon joining OIN and what this means to us Nithya Ruff also added “By joining OIN we are continuing to strengthen open source communities and helping to ensure technologies like Linux remain thriving and accessible to everyone FeedbackPlease let me know how we can improve this newsletter as well as how AWS can better work with open source projects and technologies by completing this very short survey that will take you probably less than seconds to complete Thank you so much Celebrating open source contributorsThe articles and projects shared in this newsletter are only possible thanks to the many contributors in open source I would like to shout out and thank those folks who really do power open source and enable us all to learn and build on top of what they have created So thank you to the following open source heroes Donnie Prakoso Todd Neal Tyler Lynch Thomas Roos Darko Mesaros Curtis Evans Ariel Shuper Channy Yun Jeff Barr Álvaro Hernández Heitor Lessa Abbey Fuller Noritaka Sekiyama Gonzalo Herreros Mohit Saxena Abdel Jaidi Anton Kukushkin Lucas Hanson Leon Luttenberger Zach Mitchell Ishan Gaur Kinshuk Pahare Derek Liu Pathik Shah and Raj Devnath Latest open source projectsThe great thing about open source projects is that you can review the source code If you like the look of these projects make sure you that take a look at the code and if it is useful to you get in touch with the maintainer to provide feedback suggestions or even submit a contribution Toolspg tlepg tle Trusted Language Extensions TLE for PostgreSQL pg tle is an open source project that lets developers extend and deploy new PostgreSQL functionality with lower administrative and technical overhead Developers can use Trusted Language Extensions for PostgreSQL to create and install extensions on restricted filesystems and work with PostgreSQL internals through a SQL API You can learn more about Trusted Language Extensions in the AWS News blog post New Trusted Language Extensions for PostgreSQL on Amazon Aurora and Amazon RDS where Channy Yun provides a hands on guide to getting started with this project eks node viewereks node viewer is a tool developed by Todd Neal for visualising dynamic node usage within a cluster It was originally developed as an internal tool at AWS for demonstrating consolidation with Karpenter Check out this short video from Justin Garrison that looks at this tool running Hat tip to Tyler Lynch for sharing this with me functionclarityfunctionclarity is a code integrity solution for serverless functions It allows users to sign their serverless functions and verify their integrity prior to their execution in their cloud environments FunctionClarity includes a CLI tool complemented by a verification function deployed in the target cloud account The solution is designed for CI CD insertion where the serverless function code images can be signed and uploaded before the function is created in the cloud repository Check out this blog post A New Open Source Tool that Fills a Critical Serverless Security Gap where Ariel Shuper looks at this in more detail including how you can get started dynadyna a fun project from AWS Hero Álvaro Hernández dyna is a database with limited functionality It is in reality a frontend to another database The frontend is DynamoDB API compatible That is it implements a limited subset of the same API that DynamoDB exposes Therefore it is should be compatible with DynamoDB clients and tools The main goal is to support very basic operations create table put item basic querying capabilities The backend is AWS Route a DNS service This is where data is stored and queried from As Alvaro notes in the READMEUsing DNS as a database is not a novel idea but the concept of running a database on top of Route has not been explored deep enough What do you think Get in touch with Alvaro if you try this out and let him know what you think dynamodb mass migrationsdynamodb mass migrations this repo provides a tool using AWS Step Functions Distributed Map to run massively parallel DynamoDB migrations in AWS CDK Thanks to recent accouncement of Step Functions Distributed Map we can now run of parallel executions in Step Functions This is especially useful for transforming migrating big datasets in DynamoDB aws kms xksproxy api specaws kms xksproxy api spec if you missed the announcement at re Invent AWS Key Management Service AWS KMS introduces the External Key Store XKS a new feature for customers who want to protect their data with encryption keys stored in an external key management system under their control This capability brings new flexibility for customers to encrypt or decrypt data with cryptographic keys independent authorisation and audit in an external key management system outside of AWS This repo contains the specification an example XKS client and some test clients There is also a link to the launch blog post to help get you started visual asset management systemvisual asset management system VAMS for short is a purpose built AWS native solution for the management and distribution of specialised visual assets used in spatial computing VAMS offers a simplified solution for organisations to ingest store and manage visual assets in the cloud which empowers any user with a web browser to upload manage visualise transform and retrieve visual assets Existing workflows that leverage both custom code and pre built or third party applications can also be migrated to VAMS and ran in the AWS cloud as opposed to being limited by the on premise capacity available VAMS is customisable and expandable with option of being further tailored to specific use cases by development teams aws cloud auto root volume resizeaws cloud auto root volume resize this is a project that I think a lot of folk including myself who comes up against this every time I provision a new environment will find useful Cloud provides a consistent environment for development teams that allows for ease of development by easily integrating with AWS However when launching a Cloud instance environment no options are provided that will allow for adjusting the size of the root volume and the environment will launch using the default size of GiB This limited size can prove cumbersome if teams start development work in the Cloud instance environment without realising this storage space is limited without intervention This solution allows for a near seamless integration with the existing Cloud instance environment launch process but utilising an optional tag cloud root volume size to indicate the desired root volume size in GiB fast differential privacyfast differential privacy is a library that allows differentially private optimisation of PyTorch models with a few additional lines of code It supports all PyTorch optimisers popular models in TIMM torchvision HuggingFace up to supported modules multiple privacy accountants and multiple clipping functions The library has provably little overhead in terms of training time and memory cost compared with the standard non private optimisation aws organizations alternate contacts management via csvaws organizations alternate contacts management via csv Nowadays customers have several linked accounts in their AWS Organizations These linked accounts might require different alternate contacts for many reasons and keeping such contacts updated is fundamental Unfortunately populating such contacts might be a complex and time consuming activity Customers would like to fill in their AWS linked accounts alternate contacts in a simple and quick way closer to their daily way of working like exporting to a CSV file modifying it keeping the original formatting and importing the updated contacts from the management account This is what the script does The script leverages on AWS CLI and AWS CloudShell to enable the AWS Organizations management account to easily export all the linked accounts alternate contacts to a regular CSV file Then the file can be integrated or updated and uploaded again migration hadoop to emr tco simulatormigration hadoop to emr tco simulator this repo provides you with help if you are looking to move off self managed Hadoop and migrate onto a managed service like Amazon EMR This tool may be useful when examining and estimating the cost of migration so well worth checking out aws medialive channel orchestratoraws medialive channel orchestrator this repository contains sample code to deploy a web app that can be used to simplify the management of AWS MediaLive Channels Supported functionality includes starting stopping channels input switching motion graphic overlays and much more If you use AWS MediaLive Channels then this repo is something you should check out amazon gamelift testing toolkitamazon gamelift testing toolkit this repo provides a test harness and visualisation tool for Amazon GameLift and Amazon GameLift FlexMatch The toolkit lets you visualise your GameLift infrastructure launch virtual players and iterate upon your FlexMatch rule sets with the FlexMatch simulator Detailed docs show you how to deploy and use this project Demos Samples Solutions and Workshopsaws music genre classificationaws music genre classification is a Jupyter Notebook that connects to the Registry of Open Data on AWS to show music genre classification You can run this locally or use AWS SageMaker Studio Lab this does not require an AWS account realtime toxicity detectionrealtime toxicity detection this repository contains a complete solution for detecting toxicity across voice and text chats cost efficiently and at scale in near real time It makes use of a number of AWS services including Amazon SageMaker Amazon Cognito AWS Lambda AWS Amplify and Amazon Transcribe msk powered financial data feedmsk powered financial data feed this sample application demonstrates how to publish a real time financial data feed as a service on AWS It contains the code for a data provider to send streaming data to its clients via an Amazon MSK cluster Clients can consume the data using a Kafka client SDK Detailed instructions on how to get this setup are provided in the README as well as plenty of examples in the EXAMPLES folder aws to azure bgp vpnaws to azure bgp vpn this Terraform module allows you to configure a BGP VPN Gateway between AWS and Microsoft Azure Check the docs for requirements and constraints but if you are looking to build networking across Clouds this is going to be of interest terraform eksblueprints tetrate istio addonterraform eksblueprints tetrate istio addon provides sample code on how you can deploy Istio and Envoy into your Amazon EKS environments To help you along the way check out Automate Istio Enabled Amazon EKS Cluster Deployment with Tetrate s EKS Blueprints Add On AWS and Community blog postsApache SparkCloud Shuffle Storage Plugin for Apache Spark is a new open source project under the Apache license that allows you to independently scale storage in your Spark jobs without adding additional workers With this plugin you can expect jobs processing terabytes of data to run much more reliably You can download the binaries and run them on any Spark environment The new plugin is open cloud comes with out of the box support for Amazon S and can be easily configured to use other forms of cloud storage such as Google Cloud Storage and Microsoft Azure Blob Storage To find out more check out this must read post Introducing the Cloud Shuffle Storage Plugin for Apache Spark where Noritaka Sekiyama Gonzalo Herreros and Mohit Saxena help get you started hands on DeeQuA hot topic over the past months that I have been hearing in various data communities is that around data quality Deequ is an open source library built on top of Apache Spark for defining unit tests for data Built on top of the open source DeeQu framework AWS Glue Data Quality provides a managed serverless experience to help you evaluate and monitor the quality of your data when you use AWS Glue To find out more on how to get started check out Jeff Barr s excellent post on the topic Join the Preview AWS Glue Data Quality hands on AWS Lambda SnapstartOne of the biggest announcements during re Invent was that of AWS Lambda SnapStart a new performance optimisation developed by AWS that can significantly improve the startup time for applications This feature delivers up to x faster function startup times for latency sensitive Java applications SnapStart is made possible by several pieces of open source work including Firecracker Linux CraC OpenSSL and more It is always interesting to see how these open source building blocks are combined to create great innovations like this Dive deeper by reading the post Starting up faster with AWS Lambda SnapStart on what those open source projects are and how they combine to make this all work deep dive Amazon Security LakeLaunched earlier this year the Open Cybersecurity Schema Framework is an open source project delivering an extensible framework for developing schemas along with a vendor agnostic core security schema Amazon Security Lake is a purpose built service that supports data in this format and automatically centralises an organisation s security data from cloud and on premises sources into a purpose built data lake stored in your account To find out more check out Preview Amazon Security Lake A Purpose Built Customer Owned Data Lake Service where Channy Yun looks closer at OCSF and Amazon Security Lake hands on AWS SDK for pandasAWS SDK for pandas is an open source library that extends the popular Python pandas library enabling you to connect to AWS data and analytics services using pandas data frames I love this project and use it frequently in my demos At re Invent it was announced that AWS SDK for pandas now supports Ray and Modin enabling you to scale your pandas workflows from a single machine to a multi node environment with no code changes Check out the blog post Scale AWS SDK for pandas workloads with AWS Glue for Ray where Abdel Jaidi Anton Kukushkin Lucas Hanson and Leon Luttenberger walk you through this update hands on RayRay is an open source unified compute framework that makes it simple to scale AI and Python workloads In this post Introducing AWS Glue for Ray Scaling your data integration workloads using Python Zach Mitchell Ishan Gaur Kinshuk Pahare and Derek Liu provide an introduction to AWS Glue for Ray and shows you how to start using Ray to distribute your Python workloads hands on Other posts and quick readsManaging Docker container lifecycle with AWS IoT Greengrass an interesting way on how to use AWS IoT Greengrass to control a Docker container s lifecycle using the AWS IoT Core MQTT topic and uses the message contents to execute commands against the Docker daemon with the Docker SDK for Python hands on Gain visibility into your Amazon MSK cluster by deploying the Conduktor Platform looks at how you can use Conduktor to help you solve Apache Kafka issues end to end with solutions for testing monitoring data quality governance and security hands on Managing Pod Security on Amazon EKS with Kyverno shows you how you can augment the Kubernetes Pod Security Admission PSA and Pod Security Standards PSS configurations with Kyverno hands on Managing access to Amazon Elastic Kubernetes Service clusters with X certificates is walk through on how to use X certificates as the root of trust for obtaining temporary AWS credentials to access resources in the Amazon EKS Cluster hands on AWS Thinkbox Deadline adds support for Rez looks at Rez an open source cross platform package manager and how the deep integration into AWS Thinkbox Deadline allows customers to build dynamically resolved pipelines that can be executed in an identical way on their render farm whether that be on premises or in the cloudNew for Amazon Redshift General Availability of Streaming Ingestion for Kinesis Data Streams and Managed Streaming for Apache Kafka looks at how you can now natively ingest hundreds of megabytes of data per second from Apache Kafka Amazon MSK into an Amazon Redshift materialised view and query it in seconds hands on Simplify managing access to Amazon ElastiCache for Redis clusters with IAM shows you how to use your IAM identity to authenticate and access an ElastiCache for Redis cluster hands on Introducing the Amazon Braket Algorithm Library walks you through this open source GitHub repository providing researchers ready to use Python implementations for a set of quantum algorithms on Amazon BraketIntroducing new MQTTv features for AWS IoT Core to help build flexible architecture patterns looks at how AWS IoT Core support of MQTTv features help enhance communications of large scale device deployments and innovate device messaging patterns hands on Use AWS CDK v with the AWS Amplify CLI extensibility features Preview shows you how with v beta of the Amplify CLI you can now use AWS CDK v to extend or modify your Amplify backend stack hands on Quick updatesterraform provider awsThis popular Terraform module now enables you to configure and deploy AWS Neptune Global clusters You can view the release notes here Apache Hudi Apache Iceberg Delta LakeAWS Glue for Apache Spark now supports three open source data lake storage frameworks Apache Hudi Apache Iceberg and Linux Foundation Delta Lake These frameworks allow you to read and write data in Amazon Simple Storage Service Amazon S in a transactionally consistent manner AWS Glue is a serverless scalable data integration service that makes it easier to discover prepare move and integrate data from multiple sources This feature removes the need to install a separate connector and reduces the configuration steps required to use these frameworks in AWS Glue for Apache Spark jobs These open source data lake frameworks simplify incremental data processing in data lakes built on Amazon S They enable capabilities including time travel queries ACID Atomicity Consistency Isolation Durability transactions streaming ingestion change data capture CDC upserts and deletes Amazon EMRWith Amazon EMR release you can now use Amazon Elastic Compute Cloud Amazon EC instances such as Ci Mi Ii Ri and Rid which use the third generation Intel Xeon scalable processors Using these new instances with Amazon EMR improves cost performance by an additional over previous generation instances To dive deeper into this check out the blog post Amazon EMR launches support for Amazon EC Ci Mi Ii Ri and Rid instances to improve cost performance for Spark workloads by Apache IcebergAmazon SageMaker Feature Store now supports the ability to create feature groups in the offline store in Apache Iceberg table format The offline store contains historical ML features organised into logical feature groups and is used for model training and batch inference Apache Iceberg is an open table format for very large analytic datasets such as the offline store It manages large collections of files as tables and supports modern analytical data lake operations optimised for usage on Amazon S Ingesting data especially when streaming can result in a large number of small files which can negatively impact query performance due the higher number of file operations required With Iceberg you can compact the small data files into fewer large files in the partition resulting in significantly faster queries This compaction operation is concurrent and does not affect ongoing read and write operations on the feature group If you chose the Iceberg option when creating new feature groups SageMaker Feature Store will create the Iceberg tables using Parquet file format and register the tables with the AWS Glue Data Catalog Apache SparkAmazon Athena now supports Apache Spark a popular open source distributed processing system that is optimised for fast analytics workloads against data of any size Athena is an interactive query service that helps you query petabytes of data wherever it lives such as in data lakes databases or other data stores With Amazon Athena for Apache Spark you get the streamlined interactive serverless experience of Athena with Spark in addition to SQL You can build interactive Apache PySpark applications using a simplified notebook experience in the Athena console or through Athena APIs With Athena interactive Spark applications start in under a second and run faster with our optimised Spark runtime so you spend more time on insights not waiting for results As Athena takes care of managing the infrastructure and configuring Spark settings you can focus on your business applications Dive deeper into this launch by reading the post Explore your data lake using Amazon Athena for Apache Spark where Pathik Shah and Raj Devnath show how you can use Athena for Apache Spark to explore and derive insights from your data lake hosted on Amazon Simple Storage Service Amazon S My fellow Developer Advocate Donnie Prakoso also put something together so check out his post New ーAmazon Athena for Apache Spark where he shows you how you can get started Amazon EMR announces Amazon Redshift integration with Apache Spark This integration helps data engineers build and run Spark applications that can consume and write data from an Amazon Redshift cluster Starting with Amazon EMR this integration is available across all three deployment models for EMR EC EKS and Serverless You can use this integration to build applications that directly write to Redshift tables as a part of your ETL workflows or to combine data in Redshift with data in other source Developers can load data from Redshift tables to Spark data frames or write data to Redshift tables Developers don t have to worry about downloading open source connectors to connect to Redshift OpenZFSAmazon FSx for OpenZFS now offers a new generation of file systems that doubles the maximum throughput and IOPS performance of the existing generation and includes a high speed NVMe cache Amazon FSx for OpenZFS provides fully managed cost effective shared file storage powered by the popular OpenZFS file system The new generation of FSx for OpenZFS file systems provides two performance improvements over the existing generation First new generation file systems deliver up to IOPS and GB s throughput for both reads and writes to persistent SSD storage Second they include up to TB of high speed NVMe storage that automatically caches your most recently accessed data making that data accessible at over a million of IOPS and with latencies of a few hundred microseconds With these new generation file systems you can power an even broader range of high performance workloads like media processing rendering financial analytics and machine learning with simple highly performant NFS accessible storage RayAWS Glue for Ray is a new engine option on AWS Glue Data engineers can use AWS Glue for Ray to process large datasets with Python and popular Python libraries AWS Glue is a serverless scalable data integration service used to discover prepare move and integrate data from multiple sources AWS Glue for Ray combines that serverless option for data integration with Ray ray io a popular new open source compute framework that helps you scale Python workloads You pay only for the resources that you use while running code and you don t need to configure or tune any resources AWS Glue for Ray facilitates the distributed processing of your Python code over multi node clusters You can create and run Ray jobs anywhere that you run AWS Glue ETL extract transform and load jobs This includes existing AWS Glue jobs command line interfaces CLIs and APIs You can select the Ray engine through notebooks on AWS Glue Studio Amazon SageMaker Studio Notebook or locally When the Ray job is ready you can run it on demand or on a schedule MySQLA couple of updates for MySQL users that should make you happy Improvements on both READ and WRITES of data Amazon Relational Database Service Amazon RDS for MySQL now supports Amazon RDS Optimized Reads for up to faster query processing compared to previous generation instances Optimized Read enabled instances achieve faster query processing by placing temporary tables generated by MySQL on the local NVMe based SSD block level storage that s physically connected to the host server Complex queries that utilize temporary tables such as queries involving sorts hash aggregations high load joins and Common Table Expressions CTEs can now execute up to faster with Optimized Reads on RDS for MySQL Amazon RDS Optimized Reads is available by default on RDS for MySQL versions and higher on Intel based Md and Rd instances and AWS Graviton based Mgd and Rgd database DB instances Rd and Md DB instances provide up to GiB of NVMe SSD based instance storage for low latency high random I O and sequential read throughput Mgd and Rgd DB instances are built on the AWS Nitro System and provide up to GiB of NVMe based SSD storage and up to Gbps of network bandwidth Amazon RDS Optimized Reads for Amazon RDS for MySQL is available today on Md Rd Mgd and Rgd instances in the same AWS Regions where these instances are available Amazon Relational Database Service Amazon RDS for MySQL now supports Amazon RDS Optimized Writes With Optimized Writes you can improve write throughput by up to x at no additional cost This is especially useful for RDS for MySQL customers with write intensive database workloads commonly found in applications such as digital payments financial trading and online gaming In MySQL you are protected from data loss due to unexpected events such as a power failure using a built in feature called the “doublewrite buffer But this method of writing takes up to twice as long consumes twice as much I O bandwidth and reduces the throughput and performance of your database Starting today Amazon RDS Optimized Writes provide you with up to x improvement in write transaction throughput on RDS for MySQL by writing only once while protecting you from data loss and at no additional cost Optimized Writes uses the AWS Nitro System to reliably and durably write to table storage in one step Amazon RDS Optimized Writes is available as a default option from RDS for MySQL version and above and on db ri and db rb database instances You can dive deeper by checking out Jeff Barr s post New Amazon RDS Optimized Reads and Optimized Writes Videos of the weekAWS Lambda PowertoolsHeitor Lessa provided one of the best and most eagerly anticipated sessions at re Invent AWS Lambda Powertools Lessons from the road to million downloads In this session Heitor talked about the current state of Lambda Powertools how this growth was supported key lessons learned in the past two years and what s next on the horizon LogShellThis was my top recommendation for re Invent attendees and I am super happy that the video is already available for everyone to watch Abbey Fuller is your speaker and you will learn about the response to LogShell from initial notification to hot patch fleet scanning and customer communications Open Source tools on AWSDarko Mesaros and Curtis Evans have a look at open source tools that can help make your AWS adventure easier See something for security and permissions something for cost management and a few more things for building in the cloudーtools like Infracost IAMLive and more YoctoBitBake is a make like build tool with the special focus of distributions and packages for embedded Linux cross compilation Manually upgrading bitbake recipes often and testing them is time consuming Thomas Roos talks about our approach to automate the Yocto layer maintenance of meta aws with cloud managed services technologies Maintenance means upgrading software versions testing committing bitbake recipes and back porting of changes from master to release branches You can also call it CI CD for Yocto layers Build on Open SourceFor those unfamiliar with this show Build on Open Source is where we go over this newsletter and then invite special guests to dive deep into their open source project Expect plenty of code demos and hopefully laughs We have put together a playlist so that you can easily access all seven of the other episodes of the Build on Open Source show Build on Open Source playlist Events for your diaryOpenSearchEvery other Tuesday pm GMTThis regular meet up is for anyone interested in OpenSearch amp Open Distro All skill levels are welcome and they cover and welcome talks on topics including search logging log analytics and data visualisation Sign up to the next session OpenSearch Community Meeting Stay in touch with open source at AWSI hope this summary has been useful Remember to check out the Open Source homepage to keep up to date with all our activity in open source by following us on AWSOpen	2022-12-12 08:44:16
海外TECH	DEV Community	Unexpected token in JSON at position 0 error	https://dev.to/refine/unexpected-token-in-json-at-position-0-error-1okm	Unexpected token in JSON at position errorAuthor Chukwuka Reuben Introduction This post aims to address the Unexpected token in JSON at position error message We will look into the various possible causes of this message and suggest methods to rectify it Steps we ll cover What is JSON What does the Unexpected token lt in JSON at position error mean Different Reasons Why You Might Have This Error and Their Fixes Hitting Any API endpoint that does not exist Spelling ErrorForgetting to stringify your object What is JSON JSON which stands for Javascript Object Notation can be said to be a lightweight format for storing and transporting data it is used often when data is sent from a server to a webpage If you have ever utilized API endpoints in your projects before there s a very high chance that JSON is being used to store and transport data between your web page and servers Let us quickly examine the utilization of JSON for transporting and storing data We don t need to look too far since the local storage on our internet browsers can function as servers The codeblock below shows how JSON can be used to transport data between local storage and the web page localStorage setItem list JSON stringfy list JSON parse localStorage getItem list Now that we are aware of what JSON is and how it can be applied let us move on to resolving the Unexpected token in JSON at position error message What does the Unexpected token lt in JSON at position error mean In very simple language Unexpected token lt in JSON at position indicates that you are parsing something else that is not JSON as JSON To prove my point I will attempt to reproduce the mistake Go to your browser console and execute this code snippet JSON parse undefined The code snippet above will produce this type of error Why because undefined is not JSON but we have tried to parse it as JSON There s something I would like you to note before we proceed The actual Unexpected token in JSON at position message may vary depending on what your server generates however the fundamental reason remains the same you are attempting to parse something that is not JSON as JSON Below are some of the different forms in which the error message could be presented is not a valid JSON at JSON parse Unexpected token lt lt DOCTYPE is not valid JSON Unexpected token o not found is not valid JSON Unexpected token o object obj is not valid JSON and so on So going forward I ll be using the general name JSON parse unexpected token to refer to this error Now that we know what the JSON parse unexpected token error means let us proceed to know the different reasons why you might have this error and also look into ways to fix them Different Reasons Why You Might Have This Error and Their Fixes In this section of this article reasons and their fixes will be listed Hitting Any API endpoint that does not exist This is one of the most common causes of this error and this tends to occur during the fetch request in javascript As you might have already assumed yes it occurs when you re trying to parse an endpoint result that is not JSON as JSON In this part of the article we will consider two brief cases one to obtain a valid endpoint and show the outcome and the other to retrieve an endpoint that doesn t exist so we can reproduce the error message Example In this example I ve used the JSON endpoints from a place where you can get fake JSON data to use during development I ve picked a valid endpoint from this site and went ahead to call the javascript fetch method on it check out the code snippet and its result below fetch then res gt res json then json gt console log json Using the code snippet above I want to clarify that JSON parse is being done by res json under the hood The image above shows that we got a valid JSON response now let us move to the second example Example fetch then res gt res json then json gt console log json that has been used as our API is an endpoint that I made up so it is not a valid API endpoint and as you know parsing it will be you trying to parse something that isn t JSON as it is not a formatted JSON How To Fix Make sure you are using a valid API endpoint To make sure you are using a valid JSON endpoint use JSONFORMATTER to verify your endpoints before using it Always use the try and catch within your fetch method or function to prevent your app from crashing Spelling ErrorThis is so much similar to hitting the wrong API only that you might have been pretty sure that the API endpoint exists Spelling error tends to happen due to typographical error or maybe you don t know what the correct spellings are Spelling errors do not apply only to API endpoints they can also occur while attempting to fetch information from your local storage and lead to the JSON parse unexpected token error message showing up How To Fix Check and proofread well before hitting the API Make sure you verify your API before hitting it use JSONFORMATTER Use the try and catch method in your function to prevent your app from crashing Building a side project Meet the headless React based solution to build sleek CRUD applications With refine you can build complex projects without having advanced frontend skills Try refine to rapidly build your next CRUD project whether it s an admin panel dashboard internal tool or storefront Forgetting to stringify your object If we don t use the JSON stringify technique to convert our object into a string before sending it to a server then we may encounter the error JSON parse unexpected token This raises the question why is it necessary to transform our object into a string before sending it to a server When sending data to a web server the data has to be a string and to convert a javascript object to a string JSON stringify does the trick We are going to take two quick examples in this section example will represent the problem and example will be the solution Example Note Local storage will stand as our servers in this example I have a list of todos that I have written on my web page I wish for them to stay even after I have reloaded my page how do I make that happen I have to send those lists as data to my server and then to retrieve them whenever I reload the page localStorage setItem list list In the code snippet that I have provided I have sent my data to the server without converting the object to a string using JSON stringify Let s take a look at the consequence this will have on our page below is the code snippet for retrieving the list and an image of the result const getLocalStorage gt let list localStorage getItem list if list return list JSON parse localStorage getItem list else return The error indicates that I m trying to parse an object and it s not a valid JSON Example The fix All we have to do to fix this error is to stringify our list before sending it to the server localStorage setItem list JSON stringify list The code snippet above will fix the error In general it is always a good idea to carefully check your JSON data for any syntax errors before attempting to parse it This will help to ensure that your code is able to properly handle the JSON data and avoid any errors like the Unexpected token in JSON at position error	2022-12-12 08:41:16
海外TECH	DEV Community	Use session middleware to implement distributed session solution based on Redis	https://dev.to/justlorain/use-session-middleware-to-implement-distributed-session-solution-based-on-redis-5b65	Use session middleware to implement distributed session solution based on Redis IntroductionThe main content of this post is to introduce a bizdemo hertz session The link to the demo is here This demo is designed to help users quickly get started with the Session middleware and CSRF middleware of the Hertz framework and to show the distributed Session solution based on Redis If you don t know what Hertz is then you can check out my previous articles which will help you get started with this Golang HTTP framework quickly The main features of the hertz session Use thrift IDL to define HTTP interfaceUse hz to generate codeUse hertz contrib sessions to store sessionsUse hertz contrib csrf to prevent Cross Site Request Forgery attacksUse Gorm and MySQLUse AdminLTE as frontend page HertzHertz is an ultra large scale enterprise level microservice HTTP framework featuring high ease of use easy expansion and low latency etc Hertz uses the self developed high performance network library Netpoll by default In some special scenarios Hertz has certain advantages in QPS and latency compared to go net In internal practice some typical services such as services with a high proportion of frameworks gateways and other services after migrating Hertz compared to the Gin framework the resource usage is significantly reduced CPU usage is reduced by with the size of the traffic For more details see cloudwego hertz How to get this demoUse the following command to get hertz session git clone cd bizdemo hertz session Project structurebiz holds the main business logic code including handler for HTTP requests dal for database operations and mw for middlewareidl stores thrift IDLpkg for some utility methods business constants errmsg template rendering etcstatic holds the frontend static files all taken from AdminLTEOther files in the root directory include the main startup file main go docker config files etc Use of middleware Session middlewareThe distributed session solution based on redis is to store the sessions of different servers in redis or redis cluster which aims to solve the problem that the sessions of multiple servers are not synchronized in the case of distributed system We store the session in Redis by using Hertz s session middleware which is initialized as follows biz mw session gofunc InitSession h server Hertz store err redis NewStore consts MaxIdleNum consts TCP consts RedisAddr consts RedisPasswd byte consts SessionSecretKey if err nil panic err h Use sessions New consts HertzSession store First connect to Redis using redis NewStore by passing the address password etcUse the Session middleware via the h Use method and pass in the storage connection object that we just returned The first parameter is the name of the Cookie which we pass in as a defined constant Once initialized we can store the user s information username in this case in the session after the user is authenticated and logged in Here s the core code for the Login Handler Session biz handler user user service go Loginsession sessions Default c session Set consts Username req Username session Save First get a Session object via sessions DefaultThe username is stored in the Session using the Set methodFinally we call Save to save the Session objectAfter the user is logged in we will store a copy of the user s information in the session object stored in Redis so the user can visit the corresponding page without logging in In this case there is only one home page index html which will be checked if the user is logged in and redirected to the login page login html if they are not logged in The core code to check if they are logged in is as follows pkg render render goh GET index html func ctx context Context c app RequestContext session sessions Default c username session Get consts Username if username nil c HTML http StatusOK index html hutils H message utils BuildMsg consts PageErr c Redirect http StatusMovedPermanently byte login html return c HTML http StatusOK index html hutils H message utils BuildMsg username string It s exactly the same as Login in step but instead of Set it s Get and without Save Finally we need to clean up the user s session when the user logs out the core code is as follows biz handler user user service go Logoutsession sessions Default c session Delete consts Username session Save Again note that you need to Save otherwise the delete operation will be invalid Session middleware encapsulates most of the complex logic that needs to be considered such as the storage of different user sessions in Redis and we only need to call a simple interface to complete the corresponding business process CSRF middlewareNext up is the use of CSRF middleware The following explanation of what CSRF attacks are is taken from Wikipedia Cross site request forgery also known as one click attack or session riding and abbreviated as CSRF sometimes pronounced sea surf or XSRF is a type of malicious exploit of a website or web application where unauthorized commands are submitted from a user that the web application trusts There are many ways in which a malicious website can transmit such commands specially crafted image tags hidden forms and JavaScript fetch or XMLHttpRequests for example can all work without the user s interaction or even knowledge Unlike cross site scripting XSS which exploits the trust a user has for a particular site CSRF exploits the trust that a site has in a user s browser In a CSRF attack an innocent end user is tricked by an attacker into submitting a web request that they did not intend This may cause actions to be performed on the website that can include inadvertent client or server data leakage change of session state or manipulation of an end user s account After reading the relevant information I understand that malicious websites use the trust of some websites on the user s browser such as the use of cookies to launch some attacks In this demo since the CSRF middleware is added later it was not considered when defining IDL at the beginning so there is only one GET request to log out after login Since GET is considered to be a safe method here we mainly use the CSRF middleware to protect the registration and login two POST form submissions Let s also take a look the initialization and usage of CSRF as follows func InitCSRF h server Hertz h Use csrf New csrf WithSecret consts CSRFSecretKey csrf WithKeyLookUp consts CSRFKeyLookUp csrf WithNext utils IsLogout csrf WithErrorFunc func ctx context Context c app RequestContext c String http StatusBadRequest errors New consts CSRFErr Error c Abort Call h Use using CSRF middlewareDefine the token with csrf WithSecretRetrieve the CSRF Token from the post form using the csrf WithKeyLookup definition default to the request headerUse csrf WithNext to skip the no login case i e no Cookie existsCustom exception handling with csrf WithErrorFunc After initialization we only need to submit the generated CSRF Token through the hidden field when submitting the form after login and the middleware will automatically help us verify whether it is valid If there is an error the exception handling function just defined will be used because this Demo uses the template rendering mode The core code is as follows in the case of registration the same goes for login pkg render render goh GET register html func ctx context Context c app RequestContext if utils IsLogout ctx c token csrf GetToken c c HTML http StatusOK register html hutils H message utils BuildMsg Register a new membership token utils BuildMsg token The core HTML template is as follows lt div gt lt input type hidden name csrf value token BuildMsg gt lt div gt After verifying that the user is logged in we can get the corresponding token and put it in the form to submit and let the middleware handle the rest Run the demoExecute the following command to run the demo docker compose upgo run then visit localhost register html to visit the register page here are the sample pages SummaryThat s all for this article I would be happy if you found this helpful Feel free to leave a comment if you have any questions Reference List	2022-12-12 08:36:17
海外TECH	DEV Community	A few personal stories from re:Invent 2022	https://dev.to/aws-builders/a-few-personal-stories-from-reinvent-2022-4g09	A few personal stories from re Invent As we are now in the post Invent phase of and over days have passed since re Invent in Las Vegas was concluded it s time for a lot of re Cap Blog posts and events I ve read so many of those major announcments articles that I ve decided to write a different type of re Cap for myself this year Sharing a few stories from my days in Las Vegas as they are as equally important as the technical announcements made by AWS Indeed it was a great conference with a lot of learnings and a lot of very interesting sessions I focused on Chalk Talks Builder s sessions and events like Gameday as these are not recorded Making new friends before re Invent kicks offMy flight this year got moved from Saturday to Friday so I had one more day to get over Jet Lag On Friday I spend a good time shopping and besides that met with Oliver vor dinner On saturday morning early morning I looked at the AWS Community Builders Slack and found out that Traian set up a spontanous breakfast for the Jet lagged folks and I ended up sitting over two hours with different parties having fun chatting and getting to know people It was exciting to meet Rafael who had been our Solutions Architect for a while for the first time in person without planning it It was also great to meet Heitor in person the person that owns the Lambda Power Tools at AWS His talk is now on Youtube and I would encourage you to listen to it if you are interested in Open Source The rest of the saturday I spend with Markus who shared so much Las Vegas knowledge with me that I think my brain is still burning and I would not dare to claim remembering more than half of what we discussed but it was a great saturday which ended with meeting Philipp for dinner at The Cheasecake factory I kicked off the sunday with a lot of excitement about my very first talk at re Invent final technical check in the Speaker Ready Room for my slide deck That needed to be early morning because afterwards I had planned to go out hiking with fellow AWS Community Builders Hiking across time zones and culturesDefinately one of my highlights this year The ever first AWS Community Builders pre Invent Hiking Trip Thanks to everyone that joined Oliver Richard Jenn Ganesh Traian Pubudu Niklas It was great to see how we supported each other had great conversations and all managed to get across different challenges we had to fight Thank you Oliver and kreuzwerker for the amazing video Traian you re my hero Congratulations on finishing off the hike with us with Thats an achievement noone can take away It was fascinating to meet you all for the first time and notice that we get a long well without ever meeting before That s the power of the AWS community We got back at pm after a over hours hike just in time to get our AWS re Invent badges and to meet other AWS Community Builders from around the globe for a great dinner Lily amp Jason thanks for joining us that really made me happy Welcome to re Invent Kicking off re Invent with a GameDay with a great team an Jeff BarrI decided to kick off my re Invent on Monday with a GameDay which is a fascinating opportunity for gamified learning On sunday during our hike I had aligned with Niklas to form a team together and the other two team members JaeJun and Martin we met in the morning We had great fun ended up th even tho Jeff Barr distracted us for some time as we won him on our table with a quizz It was great meeting him in person and I can tell you He is a human as we are even if his Newsblog is legendary Meeting people from the AWS CommunityI had so much great hours in Las Vegas thanks for the time spend together everyone that I ve met Stefanie Oliver Manuel Mike Stefan Philipp Markus Thorsten and others from the german community Finally met Danielle and Matt in person Another of my highlights The Community Builders Mixer and the User Group Leader Mixer where both great events to get to know each other better and network with great people from everywhere in the world I met so many people that I had been interacting with in written Slack Twitter LinkedIn it was a blast for myself Famous Jenga game with AWS Heros so much fun Speaking at re Invent my DevChatAs I ve already shared before this year I had the opportunity to speak at re Invent COM Using CDK pipelines in Java to build a multi platform Flutter applicationThanks for everyone that made this possible Ernesto Shantavia Lilly Jason Maria It was my biggest honor to share my experiences and my open source initiative Looking forward to keep sharing knowledge Announcing Amazon CodeCatalystWith the announcement of Amazon CodeCatalyst the conference brought for me a new service that I am eager to use and try out as I am very much interested in CI CD on AWS This was for me definately the most exciting anouncement of re Invent and I had a lot of interest to talk to the service team product managers and others after the service had been announced I m looking forward to share more about that as I get to play around with it more Flying outOn saturday my trip to re Invent was over and it ended as my re Invent trip began meeting AWS interested persons at the airport Thanks Maria for the introduction and with great conversations with Oliver on the way back to Frankfurt Thanks to everyone that I met and talked to at re Invent you really made this conference be a different one for me than it was before I m looking forward to hopefully meet all of you again in	2022-12-12 08:14:43
海外TECH	CodeProject Latest Articles	A Sonos Speaker System Remote for the TTGO T-Display v1	https://www.codeproject.com/Articles/5348831/A-Sonos-Speaker-System-Remote-for-the-TTGO-T-Displ	sonos	2022-12-12 08:21:00
金融	RSS FILE - 日本証券業協会	インターネット取引に係るシステム障害件数	https://www.jsda.or.jp/shiryoshitsu/toukei/inter/index.html	取引	2022-12-12 09:00:00
海外ニュース	Japan Times latest articles	With FIFA World Cup rights, Abema looks to score livestreaming win	https://www.japantimes.co.jp/news/2022/12/12/business/abema-world-cup-rights/	With FIFA World Cup rights Abema looks to score livestreaming winThe platform s purchase of broadcast rights for all games ーpossibly costing as much as billion million ーis part of a strong	2022-12-12 17:13:47
海外ニュース	Japan Times latest articles	‘War’ selected as kanji of the year amid international conflicts	https://www.japantimes.co.jp/news/2022/12/12/national/2022-kanji-war/	War selected as kanji of the year amid international conflictsThe kanji for sen meaning war battle or match was picked as the single most representative Chinese character corresponding to the social mood in Japan	2022-12-12 17:11:33
ニュース	BBC News - Home	Children who fell into icy lake near Birmingham remain critical	https://www.bbc.co.uk/news/uk-england-birmingham-63941050?at_medium=RSS&at_campaign=KARANGA	birmingham	2022-12-12 08:45:27
ニュース	BBC News - Home	Four still missing after flats blast kills five	https://www.bbc.co.uk/news/world-europe-jersey-63940492?at_medium=RSS&at_campaign=KARANGA	blast	2022-12-12 08:17:29
ニュース	BBC News - Home	Majidreza Rahnavard: Iran carries out second execution over protests	https://www.bbc.co.uk/news/world-middle-east-63939428?at_medium=RSS&at_campaign=KARANGA	rahnavard	2022-12-12 08:29:54
ニュース	BBC News - Home	UK weather: Dozens take refuge in pub amid heavy snowfall	https://www.bbc.co.uk/news/uk-england-63941122?at_medium=RSS&at_campaign=KARANGA	burwash	2022-12-12 08:50:40
ニュース	BBC News - Home	Ministers to hold emergency Cobra meeting amid wave of strikes	https://www.bbc.co.uk/news/uk-63939396?at_medium=RSS&at_campaign=KARANGA	border	2022-12-12 08:16:10
ニュース	BBC News - Home	UK economy will get worse before it gets better, warns chancellor	https://www.bbc.co.uk/news/business-63891327?at_medium=RSS&at_campaign=KARANGA	challenging	2022-12-12 08:54:58
ニュース	BBC News - Home	Pakistan v England: Tourists win thriller in Multan to seal series	https://www.bbc.co.uk/sport/cricket/63936216?at_medium=RSS&at_campaign=KARANGA	historic	2022-12-12 08:48:34
ビジネス	ダイヤモンド・オンライン - 新着記事	中国アパレルのシーイン、マーケットプレイスへの移行検討 - WSJ発	https://diamond.jp/articles/-/314417	移行	2022-12-12 17:03:00
ニュース	Newsweek	ジョージ・クルーニー、「SNSをしないこと」がスターであり続けるための秘訣	https://www.newsweekjapan.jp/stories/culture/2022/12/sns-70.php	ジョージ・クルーニー、「SNSをしないこと」がスターであり続けるための秘訣SNSの利用が当たり前となった今も、頑なにソーシャルメディアから距離を置くハリウッドスターもいる。	2022-12-12 17:05:00
ビジネス	プレジデントオンライン	中国人が｢パブロンゴールド｣を買い占める…中国のコロナ政策の転換で､日本の風邪薬が売り切れるナゾ - ｢業者が億単位で買い占めている｣とのウワサも	https://president.jp/articles/-/64419	中国政府	2022-12-12 18:00:00
IT	週刊アスキー	抽選でオリジナルグッズをプレゼント！　東京都交通局、都営交通の駅などをめぐるクイズラリーを開催	https://weekly.ascii.jp/elem/000/004/116/4116989/	新宿西口駅	2022-12-12 17:40:00