投稿時間:2023-05-31 01:20:16 RSSフィード2023-05-31 01:00 分まとめ(22件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
IT	気になる、記になる…	Amazon、半球型のコンパクトな形状が特徴の新型スマートスピーカー｢Echo Pop｣を本日発売	https://taisy0.com/2023/05/31/172301.html	amazon	2023-05-30 15:07:30
AWS	AWS The Internet of Things Blog	Ingesting enriched IoT data into Amazon S3 using Amazon Kinesis Data Firehose	https://aws.amazon.com/blogs/iot/ingesting-enriched-iot-data-into-amazon-s3-using-amazon-kinesis-data-firehose/	Ingesting enriched IoT data into Amazon S using Amazon Kinesis Data FirehoseIntroduction When sending data from Internet of Things IoT devices to a data lake you may need to enrich the device data payload with additional metadata in the cloud for further data processing and visualization There are multiple reasons this data might not exist in the device payload such as minimizing the device payload in …	2023-05-30 15:33:20
AWS	AWSタグが付けられた新着投稿 - Qiita	The instance profile aws-elasticbeanstalk-ec2-role associated with the environment does not exist.	https://qiita.com/ktsgmr/items/87539ecc4e015c18610f	fileawselasticbeanstalkec	2023-05-31 00:56:24
AWS	AWSタグが付けられた新着投稿 - Qiita	JAWS-UG SRE支部#6 CloudWatchでバレる「君、仕事中にyo〇tube観てたよね？」	https://qiita.com/kado__gen/items/4d8f0389973343e596c3	jawsug	2023-05-31 00:22:54
Git	Gitタグが付けられた新着投稿 - Qiita	【C#】ソースジェネレータを使ってみる	https://qiita.com/dsyuuto/items/4c13cc2234980a906863	sourcegenerator	2023-05-31 00:51:02
技術ブログ	Developers.IO	【Security Hub修復手順】[CloudFront.9] CloudFront ディストリビューションでは、カスタムオリジンへのトラフィックを暗号化する必要があります	https://dev.classmethod.jp/articles/securityhub-fsbp-remediation-cloudfront-9/	awssecurityhub	2023-05-30 15:03:10
海外TECH	MakeUseOf	Is Social Media Actually Making Us Less Social?	https://www.makeuseof.com/is-social-media-making-us-less-social/	social	2023-05-30 15:30:20
海外TECH	MakeUseOf	How to Fix the “Spotify Application Is Not Responding” Error in Windows 10 & 11	https://www.makeuseof.com/spotify-application-not-responding-windows/	windows	2023-05-30 15:15:18
海外TECH	DEV Community	Apache Kafka vs Memphis	https://dev.to/memphis_dev/apache-kafka-vs-memphis-48de	Apache Kafka vs Memphis What is Apache Kafka Apache Kafka is an open source distributed event streaming platform Based on the abstraction of a distributed commit log Kafka can handle a great number of events with functionality comprising pub sub What is Memphis dev Memphis is a next generation message broker A simple robust and durable cloud native message broker wrapped with an entire ecosystem that enables fast and reliable development of next generation event driven use cases Memphis dev enables building next generation applications that require large volumes of streamed and enriched data modern protocols zero ops rapid development extreme cost reduction and a significantly lower amount of dev time for data oriented developers and data engineers General LicenseBoth technologies are available under fully open source licenses Memphis also has a commercial distribution with added security tiered storage and more ComponentsKafka uses Apache Zookeeperfor consensus and message storage Memphis uses PostgreSQL for GUI state management only and will be removed soon making Memphis without any external dependency Memphis achieves consensus by using RAFT Message Consumption ModelBoth Kafka and Memphis use a pull based architecture where consumers pull messages from the server and long polling is used to ensure new messages are made available instantaneously Pull based architectures are often preferable for high throughput workloads as they allow consumers to manage their flow control fetching only what they need Storage ArchitectureKafka uses a distributed commit log as its storage layer Writes are appended to the end of the log Reads are sequential starting from an offset and data is zero copied from the disk buffer to the network buffer This works well for event streaming use cases Memphis also uses a distributed commit log called streams made by NATS Jetstream as its storage layer which can be written entirely on the broker s server memory or disk Memphis also uses offsets but abstracts them completely so the heavy lifting of saving a record of the used offsets resides on Memphis and not on the client Memphis also offers storage tiering for offloading messages to S compatible storage for an infinite storage time and more cost effective storage Reads are sequential Ecosystem and User Experience DeploymentKafka is a cluster based technology with a medium weight architecture requiring two distributed components Kafka s own servers brokers plus ZooKeeperservers Zookeeper adds an additional level of complexity but the community is in the process of removing the ZooKeeper component from Kafka Kafka Vanilla deployment requires a manual binary installation and text based configuration as well as config OS daemons and internal parameters Memphis has a light weight yet robust cloud native architecture and packed as a container from day one It can be deployed using any docker engine docker swarm and for production environment using helm for Kubernetes soon with operator Memphis initial config is already sufficient for production and optimizations can take place on the fly without downtime That approach enables Memphis to be completely isolated and apart from the infrastructure it deployed upon Enterprise support and managed cloud offeringEnterprise grade support and managed cloud offerings for Kafka are available from several prominent vendors including Confluent AWS MSK Cloudera and more Memphis provides enterprise support and managed cloud offering that includes features like enhanced security stream research abilities an ML based resource scheduler for better cost savings and more Self healingKafka is a robust distributed system and requires constant tune ups client made wrappers management and tight monitoring The user or operator is responsible for ensuring it s alive and works as required This approach has pros and cons as the user can tune almost every parameter which is often revealed as a significant burden One of Memphis core features is to remove frictions of management and autonomously make sure it s alive and performing well using periodic self checks and proactive rebalancing tasks as well as fencing the users from misusing the system In parallel every aspect of the system can be configured on the fly without downtime NotificationsMemphis has a built in notification center that can push real time alerts based on defined triggers like client disconnections resource depletion schema violation and more Apache Kafka does not offer an embedded solution for notifications Can be achieved via commercial offerings Message tracing aka Stream lineage Tracking stream lineage is the ability to understand the full path of a message from the very first producer through the final consumer including the trail and evolvement of a message between topics This ability is extremely handy in a troubleshooting process Apache Kafka does not provide a native ability for stream lineage but it can be achieved using OpenTelemetry or OpenLineage frameworks as well as integrating rd party applications such as datadog epsagon or using Confluent s cloud offering Memphis provides stream lineage per message with out of the box visualization for each stamped message using a generated header by the Memphis SDK Availability and Messaging Mirroring Replication Kafka Replication means having multiple copies of the data spread across multiple servers brokers This helps maintain high availability if one of the brokers goes down and is unavailable to serve the requests Memphis station replication works similarly During station topic creation the user can choose the number of replicas derived from the number of available brokers Messages will be replicated in a RAID manner across the chosen number of brokers Multi tenancyMulti tenancy refers to the mode of operation of software where multiple independent instances of one or multiple applications operate in a shared environment The instances tenants are logically isolated and often physically integrated The most famous users are SaaS type applications Apache Kafka does not natively support multi tenancy It can be achieved via complex client logic different topics and ACL As Memphis pushes to enable the next generation of applications and especially SaaS type architectures Memphis supports multi tenancy across all the layers from stations topics to security consumers and producers all the way to node selection for complete hardware isolation in case of need It is enabled using namespaces and can be managed in a unified console Storage tieringMemphis offers a multi tier storage strategy in its open source version Memphis will write messages that reached their end of st retention policy to a nd retention policy on object storage like S for longer retention time potentially infinite and post streaming analysis This feature can significantly help with cost reduction and stream auditing Permanent storageBoth Kafka and Memphis store data durably and reliably much like a normal database Data retention is user configurable per Memphis station or Kafka topic IdempotencyBoth Kafka and Memphis provide default support in idempotent producers On the consumer side in Kafka it׳s the client s responsibility to build a retry mechanism that will retransmit a batch of messages exactly once while in Memphis it is provided natively within the SDK with a parameter called maxMsgDeliveries Geo Replication Multi region Common scenarios for a geo replication include Geo replicationDisaster recoveryFeeding edge clusters into a central aggregate clusterPhysical isolation of clusters such as production vs testing Cloud migration or hybrid cloud deploymentsLegal and compliance requirementsKafka users can set up such inter cluster data flows with Kafka s MirrorMaker version a tool to replicate data between different Kafka environments in a streaming manner Memphis cloud users can create more Memphis clusters and form a supercluster that replicates data in an async manner between the clusters of streamed data security consumer groups unified management and more Features GUIMultiple open source GUIs have been developed for Kafka over the years for example Kafka UI Usually it cannot sustain heavy traffic and visualization and requires separate computing and maintenance There are different commercial versions of Kafka that among the rest provide robust GUI like Confluent Conduktor and more Memphis provides a native state of the art GUI hosted inside the broker built to act as a management layer of all Memphis aspects including cluster config resources data observability notifications processing and more Dead letter QueueDead letter queue is both a concept and a solution that is useful for debugging clients because it lets you isolate and recycle instead of drop unconsumed messages to determine why their processing doesn t succeed The Kafka architecture does not support DLQ within the broker it is the client or consumer s responsibility to implement such behavior for good and bad One of Memphis core building blocks is avoiding unexpected data loss enabling rapid development and shortening troubleshooting cycles Therefore Memphis provides a native solution for dead letter that acts as the station recycle bin for various failures such as unacknowledged messages schema violations and custom exceptions Schema ManagementThe very basic building block to control and ensure the quality of data that flows through your organization between the different owners is by defining well written schemas and data models Confluent offers Schema Registry which is a standalone component and provides a centralized repository for schemas and metadata allowing services to flexibly interact and exchange data with each other without the challenge of managing and sharing schemas between them It requires dedicated management maintenance scale and monitoring As part of its open source version Memphis presents Schemaverse which is also embedded within the broker Schemaverse provides a robust schema store and schema management layer on top of memphis broker without a standalone compute or dedicated resources With a unique and modern UI and programmatic approach technical and non technical users can create and define different schemas attach the schema to multiple stations and choose if the schema should be enforced or not In counter to Schema Registry the client does not need to implement serialization functions and every schema update takes place during producers runtime Message routingKafka provides routing capabilities through Kafka Connect and Kafka Streams including content based routing message transformation and message enrichment Memphis message routing is similar to the implementation of RabbitMQ using routing keys wildcards content based routing and more Similar to RabbitMQ it is also embedded within the broker and does not require external libraries or tools Log compactionCompaction has been created to support a long term potentially infinite record store based on specific keys Kafka supports native topic compaction which runs on all brokers This runs automatically for compacted topics condensing the log down to the latest version of messages sharing the same key At the moment Memphis does not support compaction but it will in the future Message replayThe ability to re consume committed messages Kafka does support replay by seeking specific offsets as the consumers have control over resetting the offset Memphis does not support replay yet but will in the near future Stream EnrichmentKafka with its Kafka Streams library allows developers to implement elastic and scalable client applications that can leverage essential stream processing features such as tables joins and aggregations of several topics and export to multiple sources via Kafka connect Memphis provides a similar behavior and more Embedded inside the broker Memphis users can create serverless type functions or complete containerized applications that aggregate several stations and streams decorate and enrich messages from different sources write complex functions that cannot be achieved via SQL and manipulate the schema Memphis embedded connectors frameworks will help to push the results directly to a defined sink Pull retry mechanismIn case of a failure or lack of ability to acknowledge consumed messages there should be a retry mechanism that will retry to pull the same offset or batch of messages In Kafka it is the client s responsibility to implement one Some key factors must be considered to implement such a mechanism like blocking vs non blocking offset tracking idempotency and more In Memphis the retry mechanism is built in and turned on by default within the SDK and broker During consumer creation the parameter maxMsgDeliveries will determine the number of retries the station will deliver a message if an acknowledgment does not arrive till maxAckTimeMs The broker itself records the offsets given and will expose only the unacknowledged ones to the retry request ConclusionApache Kafka is a robust and mature system with extensive community support and a proven record for high throughput use cases and users Still it also requires micro management troubleshooting can take precious time complex client implementations wrappers and often tunings that take the user s focus and resources from the main event and the most important thing it does not scale well as the organization grows and more use cases join in Wix Greyhound library is an excellent proof for the needed work on top of Kafka We call Memphis A next generation message broker because it leans towards the user and adapts to its scale and requirements not the opposite Most of the wrappers tunings management overhead and implementations needed from the client in Kafka are abstract to the users in Memphis which provides an excellent solution for both the smaller workload use cases and the more robust ones under the same system and with full ecosystem to support it It has a milage to pass but the immediate benefits already exist and will continue to evolve SourcesFollow Us to get the latest updates Github •Docs •DiscordJoin others and sign up for our data engineering newsletter	2023-05-30 15:12:59
海外TECH	DEV Community	Upgrading Old Version of Dokku	https://dev.to/samuelodan/upgrading-old-version-of-dokku-1hda	Upgrading Old Version of DokkuHi there I originally planned to include this set of instructions as part of another article but it was getting long So I decided to make it into its own post That said let s jump right in If Dokku was automatically included for you through the Digital Ocean DO Click Droplet chances are you re running an inexcusably old version of the tool version specifically It works fine for the most part but there have since been dozens of releases with numerous improvements The one improvement I m particularly interested in is Cron support because Dokku s Let s Encrypt plugin needs it to schedule cronjobs tasks for automatically renewing SSL certificates I spoke with the project s maintainer and he submitted a PR to update DO s Droplet to use the latest version He s also interested in automating the releases so this shouldn t be an issue in the future But in the meantime we ll have to upgrade our versions ourselves Confirm The Version You ve Got InstalledThe upgrade path is dependent on the current version Check it by running the following on your dokku hostdokku versionIf it s below x you ll have to first upgrade to x before x to avoid a breaking change Conversely you can go straight to the latest if you have a higher version Upgrade to Version Stop all your running apps first using on your dokku host for and newer versions usedokku ps stop all for versions between and usedokku ps stopall for versions between and usedokku quiet apps list xargs L dokku ps stop for versions versions older than usedokku quiet apps xargs L dokku ps stopThen run these two commands to upgrade on your dokku host run the three commands one after the otherrm bootstrap shwget sudo DOKKU TAG v bash bootstrap sh Upgrade to Version Next we ll upgrade to the latest version at the time of writing this article on your dokku host run the three commands one after the otherrm bootstrap shwget NP sudo DOKKU TAG v bash bootstrap shåConfirm the new version by running dokku version Restart Apps on your dokku hostdokku ps start allThat s it You re now on the latest version If you visit this page in the future you can find all the latest releases and their respective upgrade commands here Thanks for reading Till next time	2023-05-30 15:02:33
Apple	AppleInsider - Frontpage News	Data about Apple's AR headset screens has been leaked	https://appleinsider.com/articles/23/05/30/data-about-apples-ar-headset-screens-has-been-leaked?utm_medium=rss	Data about Apple x s AR headset screens has been leakedDisplay analyst Ross Young claims to have specifications for the screens in Apple s forthcoming Apple AR headset believed to be announced at WWDC Mockup of a possible Apple headset designApple hasn t announced any details of its first foray into Augmented Reality headsets and won t until at least WWDC But any headset will feature two screens one for each of the wearer s eyes and Young says he has the details Read more	2023-05-30 15:54:19
Apple	AppleInsider - Frontpage News	iPhone 16 Pro will be biggest iPhone yet, aspect ratio rumored to change	https://appleinsider.com/articles/23/05/30/iphone-16-pro-display-aspect-ratio-rumored-to-change?utm_medium=rss	iPhone Pro will be biggest iPhone yet aspect ratio rumored to changeThe iPhone Pro will not only be larger but it will also sport a new aspect ratio making the smartphone taller than the iPhone Pro and earlier generations A render of an iPhone ProSpeculation about the iPhone Pro models largely covers the display of the device with claims that it will be bigger than the iPhone Pro and its predecessors While it will allegedly be bigger it may also seem to be slightly different in terms of aspect ratio Read more	2023-05-30 15:41:19
海外TECH	CodeProject Latest Articles	Working with MSDTC...	https://www.codeproject.com/Tips/1233146/Working-with-MSDTC	troubleshooting	2023-05-30 15:32:00
海外科学	NYT > Science	How and Where to Watch Manhattanhenge Tonight	https://www.nytimes.com/article/manhattanhenge-sunset-time.html	manhattanhenge	2023-05-30 15:58:50
海外科学	NYT > Science	How Harmful Are Gas Stove Pollutants, Really?	https://www.nytimes.com/2023/05/30/climate/gas-stove-pollution-danger.html	stoves	2023-05-30 15:29:42
海外科学	NYT > Science	To Prevent Heart Attacks, Doctors Try a New Genetic Test	https://www.nytimes.com/2023/05/30/health/heart-attack-polygenic-risk-score.html	disease	2023-05-30 15:36:13
金融	◇◇　保険デイリーニュース　◇◇（損保担当者必携！）	保険デイリーニュース(05/31)	http://www.yanaharu.com/ins/?p=5203	損保ジャパン	2023-05-30 15:00:56
金融	金融庁ホームページ	「デジタル・分散型金融への対応のあり方等に関する研究会」（第11回）を開催します。	https://www.fsa.go.jp/news/r4/singi/20230530.html	Detail Nothing	2023-05-30 17:00:00
金融	金融庁ホームページ	証券監督者国際機構（IOSCO）による市中協議文書「暗号資産・デジタル資産に関する勧告案」について掲載しました。	https://www.fsa.go.jp/inter/ios/20230530/20230530.html	iosco	2023-05-30 16:00:00
ニュース	BBC News - Home	Artificial intelligence could lead to extinction, experts warn	https://www.bbc.co.uk/news/uk-65746524?at_medium=RSS&at_campaign=KARANGA	nuclear	2023-05-30 15:28:45
ニュース	BBC News - Home	Gender-critical professor determined to do Oxford talk	https://www.bbc.co.uk/news/education-65714821?at_medium=RSS&at_campaign=KARANGA	backs	2023-05-30 15:19:12
Azure	Azure の更新情報	General availability: Azure CNI powered by Cilium	https://azure.microsoft.com/ja-jp/updates/azure-cni-powered-by-cilium/	service	2023-05-30 16:00:06