投稿時間:2021-12-28 00:28:37 RSSフィード2021-12-28 00:00 分まとめ(30件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
python	Pythonタグが付けられた新着投稿 - Qiita	ヴェイパーヴェイヴ工場	https://qiita.com/o________o/items/0119a9eeb9e9b9933aff	ヴェイパーヴェイヴ工場この二つを組み合わせたら無限にヴェイパーヴェイヴできるんじゃないかなと思ったので工場を作るyoutubeライブ配信で垂れ流してみるサーバの用意サーバはOCIのalwaysfreeで作れるやつでubuntu入れた具体的にはこんなスペック配信環境OBSStudioってやつを入れたここを参考にした工場を稼働した稼働した録画だからまだ流れているようにみえるけど実際ライブ配信した時はCPU張り付いて動画配信がめっちゃ遅延した今後の展開もうちょっとスペックいいサーバ用意すれば回る気がするサムネが寂しいのでGANとかでなんか仮想画像作れないかなとか思いがある実はもうちょっといいサーバということで前に話題になったOCIのCPUでGメモリのVPSを取得した。	2021-12-27 23:47:40
Program	[全てのタグ]の新着質問一覧｜teratail（テラテイル）	Docker DesktopのKubernetesを使ってブラウザからSpring Bootアプリケーションへアクセスできない	https://teratail.com/questions/375801?rss=all	DockerDesktopのKubernetesを使ってブラウザからSpringBootアプリケーションへアクセスできないKubernetesを勉強しているものです。	2021-12-27 23:51:32
Program	[全てのタグ]の新着質問一覧｜teratail（テラテイル）	あるセクションをscrollしたときに、裏のカード（要素）を表示させたい	https://teratail.com/questions/375800?rss=all	あるセクションをscrollしたときに、裏のカード要素を表示させたい前提・実現したいこと表題の機能を反映させたく、質問します。	2021-12-27 23:29:36
Ruby	Rubyタグが付けられた新着投稿 - Qiita	【Rails】一定期間でログファイルを削除【手動は卒業】	https://qiita.com/takeda-no-nao/items/2f44d08b40eb6c5612d4	自動で削除できないのなら、あまり使う意味はない気もしてくるconfigenvironmentsdevelopmentrbこちらは削除してくれるconfigloggerLoggernewlogsamplelogdaily日付の場合は、退避されるだけで削除はされないらしいconfigloggerLoggernewlogsamplelogdailyまあ、でも日付つけてリネームしてくれるだけでも十分便利ではあるか。	2021-12-27 23:22:31
Ruby	Railsタグが付けられた新着投稿 - Qiita	【Rails】一定期間でログファイルを削除【手動は卒業】	https://qiita.com/takeda-no-nao/items/2f44d08b40eb6c5612d4	自動で削除できないのなら、あまり使う意味はない気もしてくるconfigenvironmentsdevelopmentrbこちらは削除してくれるconfigloggerLoggernewlogsamplelogdaily日付の場合は、退避されるだけで削除はされないらしいconfigloggerLoggernewlogsamplelogdailyまあ、でも日付つけてリネームしてくれるだけでも十分便利ではあるか。	2021-12-27 23:22:31
技術ブログ	Developers.IO	AWS AppSyncをちょっとだけ触ってみた	https://dev.classmethod.jp/articles/touching-aws-appsync-just-a-little/	awsappsy	2021-12-27 14:55:50
海外TECH	Ars Technica	Coffee’s health benefits aren’t as straightforward as they seem—here’s why	https://arstechnica.com/?p=1822369	different	2021-12-27 14:02:35
海外TECH	MakeUseOf	8 Types of Excel Charts and Graphs and When to Use Them	https://www.makeuseof.com/the-best-excel-charts-and-graphs/	Types of Excel Charts and Graphs and When to Use ThemGraphics are easier to grasp than text and numbers Charts are a great way to visualize numbers We show you how to create charts in Microsoft Excel and when to best use what kind	2021-12-27 14:46:38
海外TECH	MakeUseOf	What Is Clipping in Photography? How to Avoid or Fix It	https://www.makeuseof.com/clipping-in-photography-how-to-fix/	beginner	2021-12-27 14:30:12
海外TECH	MakeUseOf	How to Write a Compelling LinkedIn Summary (About Section)	https://www.makeuseof.com/how-to-write-linkedin-summary/	How to Write a Compelling LinkedIn Summary About Section The LinkedIn summary or About section should tell who you are why you do what you do and your best abilities Here s how to write one	2021-12-27 14:16:40
海外TECH	DEV Community	Bootstrap 5 - Open-source Flask Templates	https://dev.to/sm0ke/bootstrap-5-open-source-flask-templates-2l	Bootstrap Open source Flask TemplatesHello Coders This article presents a curated list with open source Flask Templates crafted on top of modern Bootstrap UI Kits and released under the MIT license on Github Source code for all products can be downloaded without a registration lock and used for hobby amp commercial or simply for eLearning activities Thanks for reading Soft UI Design Flask product pageFlask Material Kit product pageFlask Pixel Lite product pageAll products are generated with a an identical features set database and deployment support via Docker Up to date dependencies Flask SCSS compilation via GulpBootstrap designSQLite Flask SQLAlchemy ORMSession Based authentication flow login register Deployment Docker Gunicorn Nginx HerokuSupport via Github issues tracker and Discord Probably the most easier way to compile and use the products is via Docker Once we have the code downloaded locally the build is super easy Download the sources Soft UI Design git clone cd flask soft ui designStart in Docker docker compose up build Once all the above commands are executed we should be able to access the app in the browser create users and interact with the UI Soft UI Design FlaskSoft UI Design System is a Premium Bootstrap UI Kit designed by Creative Tim designed for those who like bold elements and beautiful websites Soft UI Design System Flask is a simple Flask starter with a modular codebase database authentication and deployment scripts Soft UI Design Flask LIVE DemoSoft UI Design Flask Product page Material Kit FlaskMaterial Kit is built with over frontend individual elements like buttons inputs navbars nav tabs cards or alerts giving you the freedom of choosing and combining Flask Material Kit LIVE DemoFlask Material Kit product page Pixel Flask TemplatePixel is a free fully responsive and modern Bootstrap UI Kit that will help you build creative and professional websites Use our components and sections switch some Sass variables to build and arrange pages to best suit your needs Flask Pixel Lite LIVE DemoFlask Pixel Lite Product pageThanks for reading For more resources feel free to access Flask official websiteAppSeed for support and more Flask starters	2021-12-27 14:28:32
海外TECH	DEV Community	How to use design patterns in JavaScript?	https://dev.to/hmnomaan/how-to-use-design-patterns-in-javascript-3kgj	How to use design patterns in JavaScript JavaScript is a lightweight interpreted object oriented programming language with first class functions most commonly known as a scripting language for web pagesWhat Are Design Patterns As I said before design patterns are reusable solutions to commonly occurring problems in software design Let s take a look at some of the categories of design patterns Proto patternsHow does one create a pattern Let s say you recognized a commonly occurring problem and you have your own unique solution to this problem which isn t globally recognized and documented You use this solution every time you encounter this problem and you think that it s reusable and that the developer community could benefit from it Does it immediately become a pattern Luckily no Oftentimes one may have good code writing practices and simply mistake something that looks like a pattern for one when in fact it is not a pattern How can you know when what you think you recognize is actually a design pattern By getting other developers opinions about it by knowing about the process of creating a pattern itself and by making yourself well acquainted with existing patterns There is a phase a pattern has to go through before it becomes a full fledged pattern and this is called a proto pattern A proto pattern is a pattern to be if it passes a certain period of testing by various developers and scenarios where the pattern proves to be useful and gives correct results There is quite a large amount of work and documentationーmost of which is outside the scope of this articleーto be done in order to make a fully fledged pattern recognized by the community Anti patternsAs a design pattern represents good practice an anti pattern represents bad practice An example of an anti pattern would be modifying the Object class prototype Almost all objects in JavaScript inherit from Object remember that JavaScript uses prototype based inheritance so imagine a scenario where you altered this prototype Changes to the Object prototype would be seen in all of the objects that inherit from this prototypeーwhich would be most JavaScript objects This is a disaster waiting to happen Another example similar to one mentioned above is modifying objects that you don t own An example of this would be overriding a function from an object used in many scenarios throughout the application If you are working with a large team imagine the confusion this would cause you d quickly run into naming collisions incompatible implementations and maintenance nightmares Similar to how it is useful to know about all of the good practices and solutions it is also very important to know about the bad ones too This way you can recognize them and avoid making the mistake up front Design Pattern CategorizationDesign patterns can be categorized in multiple ways but the most popular one is the following Creational design patternsStructural design patternsBehavioral design patternsConcurrency design patternsArchitectural design patternsCreational Design PatternsThese patterns deal with object creation mechanisms which optimize object creation compared to a basic approach The basic form of object creation could result in design problems or in added complexity to the design Creational design patterns solve this problem by somehow controlling object creation Some of the popular design patterns in this category are Factory methodAbstract factoryBuilderPrototypeSingletonStructural Design PatternsThese patterns deal with object relationships They ensure that if one part of a system changes the entire system doesn t need to change along with it The most popular patterns in this category are AdapterBridgeCompositeDecoratorFacadeFlyweightProxyBehavioral Design PatternsThese types of patterns recognize implement and improve communication between disparate objects in a system They help ensure that disparate parts of a system have synchronized information Popular examples of these patterns are Chain of responsibilityCommandIteratorMediatorMementoObserverStateStrategyVisitorConcurrency Design PatternsThese types of design patterns deal with multi threaded programming paradigms Some of the popular ones are Active objectNuclear reactionSchedulerArchitectural Design PatternsDesign patterns which are used for architectural purposes Some of the most famous ones are MVC Model View Controller MVP Model View Presenter MVVM Model View ViewModel In the following section we are going to take a closer look at some of the aforementioned design patterns with examples provided for better understanding	2021-12-27 14:25:33
海外TECH	DEV Community	The Ultimate Angular Resources🛠 For Developers👨‍💻	https://dev.to/theme_selection/the-ultimate-angular-resources-for-developers-2glm	The Ultimate Angular ResourcesFor Developers‍As we all know Angular is one of the leading JavaScript frameworks It is highly recommended by developers across the globe Here we are going to list down some of the amazing resources for Angular Although before we start the list let s have an overview of it What Is Angular Well Angular is a TypeScript based open source front end web application platform led by the Angular Team at Google and by a community of individuals and corporations to address all of the parts of the developer s workflow while building complex web applications Angular is a complete rewrite from the same team that built AngularJS It is a framework for building client applications in HTML and either JavaScript or a language like TypeScript that compiles to JavaScript Angular also combines declarative templates dependency injection end to end tooling and integrated best practices to solve development challenges Besides Angular empowers developers to build applications that live on the web mobile or the desktop Companies like Google Forbes PayPal Deutsche Bank UpWork The Guardian uses Angular Advantages Of Angular Effective Cross Platform Development High Quality of the Application Improved Speed and Performance Faster Development Process Readable and Testable Code More Lightweight Web Applications Efficient Problem Solving Patterns Excellent Material Design Library Features Two way Data binding Modular development structure Ease in Maintaining Dependency Injection TypeScript based superset of JavaScript High performance for Single Page Hefty Apps Offers great flexibilityNow let s check the Ultimate Angular Resources that will help you kickstart your journey as an Angular developers Ultimate Angular Resources Follwoing are some of the very useful Angular resources that can be helpful for beginners and experienced developers as well Official ResourcesSiteBlogDocumentationCheatsheetGetting Started GuideGitHub Repo Communities TwitterGitHubStackoverflow UI Libraries Well Angular component libraries will save your time and will help to develop an intriguing and spectacular product Besides these libraries also provide many useful components and tools as well Material Angular Angular UI Library Based On Material DesignAngular Material is the most commonly used angular UI framework with Angular projects It is an official component library developed by Google Besides the Angular team builds and maintains both common UI components and tools to help you build your custom components This angular UI framework also has a complete solution for all your UI needs Furthermore the CSS library in Angular Material consists of typography and other ready to use elements NG Bootstrap UI Library For Angular Based On The Bootstrap FrameworkNG bootstrap is another bootstrap based Angular UI framework that offers Bootstrap components for Angular One advantage of NG bootstrap is there are no dependencies required of rd party JavaScript Besides most of its features are the same as those of ngx bootstrap but with different project teams In addition it also contains third party JS dependencies and high testing coverage Moreover it is so convenient to access all the widgets Nebular set of essential Theme UI Kit AUTH Security modules for complex Angular applicationsNebular is a customizable Angular UI Library It has a beautiful design and the ability to adapt it to your brand easily Besides It comes with stunning visual themes a powerful theming engine with runtime theme switching Also it has the support of custom CSS properties mode Furthermore it is based on Eva Design System specifications Besides this angular UI library also offers stunning visual themes support of custom CSS properties mode and powering theming engine with runtime theme switching NG ZORRO An enterprise class UI components based on Ant Design and Angular NG ZORRO an Ant UI based library designed to provide added advantages to Angular developers Besides this library is developed in Chinese written in TypeScript and comes with define types Furthermore its popularity on GitHub stands with more than k Stars In addition it also supports OnPush mode high performance Other Useful UI Libraries ngSemantic UI components based on Semantic UIngx ui Style and Component Library for Angular and beyond prime ng Collection of rich UI components for Angular Wijmo Set of UI components for AngularFor more details check the Angular Component UI libraries Admin Templates Admin templates are one of the most used product in web development world They are very useful and saves a lots of time Admin templates offers components and features required for developing web apps Thus it is always a good choice to take help of it Following are some of the best open source and premium admin templates Open source Ngx AdminNgx Admin is a bright and straightforward admin template with a neat web design to create every experience a pleasurable one Besides Ngx Admin is a free admin dashboard template based on Angular and Eva Design System It is responsive and comes with Flexibly configurable themes with a hot reload themes included Light and dark version Purple AngularPurple Angular free template contains a beautiful and carefully drafted admin dashboard that is packed with plenty of essential components Besides this is also a simple yet decent admin template in the collection of Open Source Angular admin templates Guru Able Angular LiteGuru Able Angular Lite is an open source Angular Admin dashboard that is a completely free solution for your dashboard creation This angular admin template comes with Dark Version with optimized Google font integration high speed mature and sophisticated extremely well organized code makes guru Able Lite a completely flexible solution for any type of back end project Notus AngularNotus Angular is Free and Open Source It features multiple HTML and Angular elements and it comes with dynamic components for Angular It is based on Tailwind Starter Kit and it is built with both presentation pages and pages for an admin dashboard Also it is built with over frontend components giving you the freedom of choosing and combining All components can take variations in colors that you can easily modify using Tailwindcss classes For more admin templates check the collection of Open Source Angular Admin Templates Premium Vuexy Angular Admin Dashboard Template The best Vuexy Angular Admin Dashboard Template is the most developer friendly amp highly customizable Admin Dashboard Template It is based on Angular CLI NgBootstrap If you re a developer looking for an admin dashboard that is developer friendly rich with features and highly customizable look no further than Vuexy Besides it has followed the highest industry standards to bring you the very best admin template that is not only fast and easy to use but highly scalable Offering ultimate convenience and flexibility you ll be able to build whatever application you want with very little hassle Modern Admin Angular Bootstrap Admin TemplateModern Admin is the most complete amp feature packed Angular Bootstrap Admin Template with material components A combination of clean design and modular code structure which makes it most developer friendly amp highly customizable Angular Admin Dashboard Template This is a powerful admin dashboard template built on Angular CLI Besides it is developer friendly and rich in features We ve followed the leading industry standards to bring you the very best admin template that is not only fast and easy to use but highly scalable Offering ultimate convenience and flexibility you ll be able to build modern applications with very little hassle Apex Angular amp Bootstrap Admin TemplateApex Angular Admin Template stable with unlimited possibilities Besides The Angular CLI makes it easy to maintain this application that already works right out of the box Furthermore it includes solid amp gradient menu color options and different sizes with organized folder structure clean amp commented code charts pages components advance cards widgets and many more Also the code is super easy to understand and it comes with a starter kit that will help developers to get started quickly In addition the template is fully responsive and clean on every device and on every modern browser For more premium admin templates check the collection of Angular Admin Template Free Download Tutorials Angular Tutorial for Beginners Web Framework with Typescript CourseLearn the basics of Angular in this full course for beginners Course Contents IntroductionComponents Lifecycle hooks Text interpolationComponents Communication Component StylesNg Content Template StatementsPipes Property BindingAttribute Class amp Style and many more Angular tutorial for beginnersIn this angular version video we learn what is angular and we learn basic things about angular courses Course Covers Set up and installFiles and folder structureInterpolationAngular CLI and Important commandsComponent with Inline Style and Template and many more Angular Crash CourseLearn the fundamentals of Angular in this project based crash course Topics covered Intro amp Slides Angular CLI amp SetupFiles amp Folders Component structureProperties amp InterpolationGlobal Styles Header Component and many more Angular TutorialThis Edureka Angular Full Course video will help you understand and learn the fundamentals of Angular This Angular Tutorial is ideal for both beginners as well as professionals who want to master various concepts of Angular Topics covered in this video are IntroductionBuilding Blocks of Web DevelopmentIntroduction to AngularAngular Architecture Angular Installation amp Setup Angular CLI Files and Folders Angular ComponentsAngular TemplatesAngular Modules and many more For more tutorials check the collection of Angular Tutorials For Beginners Books Angular Projects Second EditionThis updated second edition of Angular Projects will teach you how to build efficient and optimized web applications using Angular You will learn and explore the fundamental features of the framework by creating ten different real world web apps Each app will elaborate how to integrate Angular with a different library and tool As you goes ahead you will understand how to execute popular technologies such as Angular Router Electron Scully Electron Nx monorepo tools Angular service worker NgRx and more while building an issue tracking system and many other exciting projects Angular RouterThis book is a complete description of the Angular router written by its designer It goes far beyond a how to get started guide and talks about the library in depth The mental model design constraints and the subtleties of the API everything is covered Reading this book will give you deep insights into why the router works the way it does and will make you an Angular router expert The book explores the library in depth including the mental model design constraints subtleties of the API Angular Buch German This book introduces you to the building blocks of Angular many best practices and the tools you need to use Get started with a practical introduction The Book Deals With Reactive programming with RxJS State management with Redux and NgRx Testing with Jasmine Karma and Protractor Routing guards and modular system HTTP and interceptors Form processing Dependency Injection und Services and many more Angular Projects Build Modern Web Apps by Exploring Angular As you go through the book you will learn how to implement popular technologies such as Angular Router Scully Electron Angular service worker Nx monorepo tools NgRx and more while building an issue tracking system a PWA weather application a mobile photo geotagging application a component UI library and many other exciting projects What you will learn Set up Angular applications using Angular CLI and Nx Console Create a personal blog with Jamstack and SPA techniques Build desktop applications with Angular and Electron Enhance user experience UX in offline mode with PWA techniques Make web pages SEO friendly with server side rendering Create a monorepo application using Nx tools and NgRx for state management Focus on mobile application development using Ionic Develop custom schematics by extending Angular CLI The Ng book ーA Complete Book on Angularng book is designed to teach you step by step how to create serious Angular apps from empty folder to deployment Each chapter covers a topic and we provide full code examples for every project in the book The book also describes how to write and test mobile apps with NativeScript The book covers Dependency Injection Data Architecture in Angular Data Architecture with Observables and RxJS Data Architecture with Redux Redux and TypeScript Data Architecture with ReduxOther Useful Books To Learn AngularArchitecting Angular Applications with NGRX Becoming a Ninja with Angular Developing with AngularEssential Angular IDE sIntegrated Development Environment IDE on the other hand is a program that simplifies the effort required by developers by optimizing the entire development process and help you in writing flawless code It helps in integrating the software into the system and analyze the working Angular IDEAngular IDE is highly used for faster and effective development It is a stand alone plugin that can be used with the Eclipse plugin It is built specifically for Angular but initially it was used for tsconfig json management The benefits of Angular IDE are It offers real time validation of code displays errors as you type It features auto completion of code in the project It highly uses coloring and highlights the syntax aware source It features block and full formatting with advanced settings Aptana StudioIt is an open source IDE for Angular You can add the AngularJS Eclipse extension It is suitable for many other frameworks as well as libraries It is based on Eclipse and is highly used for JavaScript development It comes both as a stand alone version and an Eclipse Plug in It also has a fully integrated Ruby amp Rails debugger to use its features to be the most professional grade product Benefits of Aptana Studio are It supports other Testing tools It has the Deployment wizard It can be quickly and thoroughly customized In built terminal ALM IDEALM IDE is a cloud based ide available for TypeScript It enables you to code quickly in TypeScript projects You can easily install it with just a single npm command It is easy to use and it removes all the barriers to use TypeScript You can even develop TypeScript projects on Raspberry π The benefits of using ALM are It makes it easy to use TypeScript Lightweight Made for TypeScriptOther Useful IDE for Angular Angular IDE by WebclipseIntelliJ IDEAVisual Studio CodeWebStorm Documentation tools Storybook The UI development environment you ll love to useCompodoc The missing documentation tool for your Angular application integrate well with npm scriptsNgModule Viz Visualize the dependencies between the NgModules in your Angular application ng app counter Count the number of Modules Lazy Modules Pipes Providers Directives Components used in an Angular application Developer tools Angular State Inspector Helps you debug Angular component state Supports Angular Ivy Augury Browser extension for debugging and profiling Angular applications CodeSandbox An online IDE and prototyping tool for rapid Angular development Bugfender A cloud service to collect logs and Angular errors in Real Time Conclusion Here we have mentioned some of the very useful Angular resources While you are working with the Angular project you ll surely need some tools UI component libraries admin templates video tutorials etc So to help you save your time we have gathered some amazing resources here so that you don t have to search here and there We are open to suggestions so do tell us which other resources can be included in the list Also don t forget to share and bookmark this collection We hope you find this collection useful	2021-12-27 14:17:28
海外TECH	DEV Community	Introduction to Batch Data Processing	https://dev.to/aws-builders/introduction-to-batch-data-processing-4k56	Introduction to Batch Data ProcessingMost analytics applications require frequent batch processing which allows them to process data in batches at varying interval For example processing daily sales aggregations by individual store and then writing that data to the data warehouse on a nightly basis can allow business intelligence BI reporting queries to run faster Batch systems must be built to scale for all sizes of data and scale seamlessly to the size of the dataset being processed at various job runs It is important for the batch processing system to be able to support disparate source and target systems process various data formats seamlessly scale out to process peak data volumes ability to orchestrate jobs using workflow provide a simple way to monitor the jobs and most importantly offer an ease of use development framework that accelerates job development Business requirements might dictate that batch data processing jobs be bound by an SLA or have certain budget thresholds Use these requirements to determine the characteristics of the batch processing architecture Architecture Options for Building an Analytics Application on AWS is a Series containing different articles that cover the key scenarios that are common in many analytics applications and how they influence the design and architecture of your analytics environment in AWS These series present the assumptions made for each of these scenarios the common drivers for the design and a reference architecture for how these scenarios should be implemented On AWS analytic services such as Amazon EMR Amazon Redshift Lake Formation Blueprints and AWS Glue family services namely Glue ETL Glue Workflows AWS Glue DataBrew AWS Glue Elastic Views allow you to run batch data processing jobs at scale for all batch data processing use cases and for various personas such as data engineer data analyst and data scientists While there are some overlapping capabilities between these services knowing the core competencies and when to use which service or services allow you to accomplish your objectives in the most effective way CharacteristicsFollowing are the key characteristics that determine how you should plan when developing a batch processing architecture Ease of use development framework This is one of the most important characteristics that truly allows personas of ETL developers and data engineers data analysts and data scientists to improve their overall efficiencies An ETL developer benefits from a hybrid development interface that helps them to use the best of bothーdeveloping part of their job and switching to writing customized complex code where applicable Data analysts and data scientists spend a lot of their time preparing data for actual analysis or capturing feature engineering data for their machine learning models You can improve their efficiencies in data preparation by adopting a no code data preparation interface which helps them to normalize and clean data up to faster compared to traditional approached to data preparation Support disparate source and target systems Your batch processing system should have the ability to support various different types of data sources and targets between relational semi structured non relational and SaaS providers When operating in the cloud a connector ecosystem can benefit you by seamlessly connecting to various sources and targets and can simplify your job development Support various data file formats Some of the commonly seen data formats are CSV Excel JSON Apache Parquet Apache ORC XML and Logstash Grok Your job development can be accelerated and simplified if the batch processing services can natively profile these various file formats infer schema automatically including complex nested structures so that you can focus more on building transformations Seamlessly scale out to process peak data volumes Most batch processing jobs experience varying data volumes Your batch processing job should have the ability to scale out to handle peak data spikes and scale back in when the job completes Simplified job orchestration with job bookmarking capability The ability to develop job orchestration with dependency management and the ability to author the workflow using API CLI and a graphical user interface allows for a robust CI CD integration Ability to monitor and alert on job failure This is an important measure for ease of operational management Having quick and easy access to job logs and a graphical monitoring interface to access job metrics can quickly help you identify errors and tuning opportunities for your job Coupling that with an event driven approach to alert on job failure will be invaluable for easing operational management Provide a low cost solution Costs can quickly get out of control if you do not plan correctly A pay as you go pricing model for both compute and authoring jobs can help you overcome hefty costs upfront and allows you to only pay for what you use instead of overpaying to accommodate for peak workloads Use automatic scaling to accommodate spiky workloads when necessary Using Spot Instances where applicable can bring your costs down for workloads where they are a good fit Reference architectureBatch data processing reference architectureBatch data processing systems typically require a persistent data store for source data When developing batch data processing applications on AWS you can use data from various sources including your on premises data stores Amazon RDS Amazon S DynamoDB and any other databases that are accessible from the cloud Data processing jobs need access to a variety of data stores to read data You can use AWS Glue connectors from the AWS Marketplace to connect to a variety of data stores such as Google BigQuery and SAP HANA You can also connect to SaaS application providers such as Salesforce ServiceNow and Google Analytics using AWS Glue DataBrew and Amazon AppFlow In addition you can always rely on the custom JDBC capability in Apache Spark and connect to any JDBC compliant data store from Amazon EMR or AWS Glue jobs Choosing the right authoring tool for the job simplifies job development and improves agility a You can use AWS Glue Studio when authoring jobs for the AWS Glue Spark runtime engine b Use AWS Glue Blueprints when you need to create a self service parametrized job for analysts and control what data the analyst is allowed to access c Use Amazon EMR notebook for interactive job development and scheduling notebook jobs against Amazon EMR d Use Amazon SageMaker Notebook when working within SageMaker ML development e Use AWS Glue DataBrew from the AWS Management Console or from a Jupyter Notebook for no code development experience f Use Lake Formation Blueprints when you need to quickly create batch data ingestion jobs to rapidly build a data lake in AWS Choosing the right processing engine for your batch jobs allows you to be flexible with managing costs and lowering operational overhead Today only Amazon EMR and Amazon Redshift offer the ability to autoscale seamlessly based on your job runtime metrics using the managed scaling feature and concurrency scaling features for read and write preview respectively Other services depicted under job processing in the reference architecture require you to specify the maximum processing capacity for your job which will be used to accommodate peak data volumes Amazon EMR and Amazon Redshift offer server based architectures while rest of the services depicted in the reference architecture are fully serverless Amazon EMR allows you to use Spot Instances for suitable workloads that can further save your costs A good strategy is to complement these processing engines to meet the business objectives of the SLA functionality and lower TCO by choosing the right engine for the right job Batch processing jobs usually require writing processed data to a target persistent store This store can reside anywhere between AWS on premises or other cloud providers You can use the rich connector interface AWS Glue offers to write data to various target platforms such as Amazon S Snowflake and Amazon OpenSearch Service You can also use the native spark JDBC connector feature and write data to any supported JDBC target All batch jobs require a workflow that can handle dependency checks to ensure no downstream impacts and have a bookmarking capability that allows them to resume where they left off in the event of a failure or at the next run of the job When using AWS Glue as your batch job processing engine you can use the native workflow capability to help you create a workflow with a built in state machine to track the state of your job across the entire workflow Glue jobs also support a bookmarking capability that keeps track of what it has processed and what needs to be processed during next run Similarly AWS Lake Formation blueprints support a bookmarking capability when processing incremental data With EMR Studio you can schedule notebook jobs When using any of the analytic processing engines you can build job workflows using an external scheduler such as AWS Step Functions or Amazon Managed Workflows for Apache Airflow that allows you to interoperate between any service including external dependencies Batch processing jobs write data output to target data store which can be anywhere within the AWS Cloud on premises or at another cloud provider You can use the AWS Glue Data Catalog to crawl the supported target databases to simplify writing to your target database Configuration notesUse AWS Glue Data Catalog as a central metastore for your batch processing jobs regardless of which AWS analytic service you use as a processing engine Batch processing jobs cater to a variety of workloads ranging from running several times an hour or day to running monthly or quarterly The data volumes vary significantly and so do the consumption patterns on the processed dataset Always work backwards to understand the business SLAs and develop your job accordingly The central Data Catalog makes it easy for you to use the right analytic service to meet your business SLAs and other objectives thereby creating a central analytic ecosystem Avoid lifting and shifting server based batch processing systems to AWS By lifting and shifting traditional batch processing systems into AWS you risk running over provisioned resources on Amazon EC For example traditional Hadoop clusters are often over provisioned and idle in an on premises setting Use AWS Managed Services such as AWS Glue Amazon EMR and Amazon Redshift to simplify your architecture using a lake house pattern and remove the undifferentiated heavy lifting of managing clustered and distributed environments Automate and orchestrate everywhere In a traditional batch data processing environment it s a best practice to automate and schedule your jobs in the system In AWS you should use automation and orchestration for your batch data processing jobs in conjunction with the AWS APIs to spin up and tear down entire compute environments so that you are only charged when the compute services are in use For example when a job is scheduled a workflow service such as AWS Step Functions would use the AWS SDK to provision a new EMR cluster submit the work and shut down the cluster after the job is complete Similarly you can use Terraform or a CloudFormation template to achieve similar functionality Use Spot Instances and Graviton based instance types on EMR to save costs and get better price performance ratio Use Spot Instances when you have flexible SLAs that are resilient to job reruns upon failure and when there is a need to process very large volumes of data Use Spot Fleet EC Fleet and Spot Instance features in Amazon EMR to manage Spot Instances Continually monitor and improve batch processing jobs Batch processing systems evolve rapidly as data source volumes increase new batch processing jobs are authored and new batch processing frameworks are launched Instrument your jobs with metrics timeouts and alarms to have the metrics and insight to make informed decisions on batch data processing system changes Hope this guide gives you an Introduction to Batch Data Processing explains the Characteristics Reference Architecture and Configuration Notes for Batch Data Processing Let me know your thoughts in the comment section And if you haven t yet make sure to follow me on below handles connect with me on LinkedInconnect with me on Twitter‍follow me on github️Do Checkout my blogs Like share and follow me for more content ltag user id follow action button background color important color fac important border color important Adit ModiFollow Cloud Engineer AWS Community Builder x AWS Certified x Azure Certified Author of Cloud Tech DailyDevOps amp BigDataJournal DEV moderator Reference Guide	2021-12-27 14:15:57
Apple	AppleInsider - Frontpage News	Best deals Dec. 27: $169 Beats Powerbeats Pro, $60 4TB hard drive, Lego Boba Fett Helmet, more!	https://appleinsider.com/articles/21/12/27/best-deals-dec-27-169-beats-powerbeats-pro-60-4tb-hard-drive-lego-boba-fett-helmet-more?utm_medium=rss	Best deals Dec Beats Powerbeats Pro TB hard drive Lego Boba Fett Helmet more The best deals for December include off Beats Powerbeats Pro off a inch LG K monitor and a TB WD Red NAS drive for Best deals for December To help you search through the continuing holiday deal chaos we ve curated some of the best deals we could find on Apple products tech accessories and other things for the AppleInsider audience Read more	2021-12-27 14:55:32
Apple	AppleInsider - Frontpage News	The first HomeKey lock & the best motion sensors on HomeKit Insider	https://appleinsider.com/articles/21/12/27/the-first-homekey-lock-the-best-motion-sensors-on-homekit-insider?utm_medium=rss	The first HomeKey lock amp the best motion sensors on HomeKit InsiderThis week on the HomeKit Insider podcast your hosts discuss the first possible sighting of Apple HomeKey Aqara s new product line and round up the best HomeKit compatible motion sensors Apple announced its new HomeKey feature with iOS but we ve yet to see any products launch with it included We may have had our first sighting as a still unannounced lock from Schlage mentions it on its since removed Home Depot listing Since the last episode Aqara has made several new product announcements such as a ceiling mounted simulated skylight Time will tell how many make their way to the U S Read more	2021-12-27 14:19:37
海外TECH	Engadget	Final Fantasy XVI delayed a half year due to COVID-19	https://www.engadget.com/final-fantasy-xvi-delayed-six-months-due-to-covid-19-142140517.html?src=rss	Final Fantasy XVI delayed a half year due to COVID Last year Square Enix unveiled Final Fantasy XVI the next chapter in the long running RPG series promising more information to come at the end of Now it seems that both the announcement and the game have been postponed Kotaku has reported quot When we last spoke I promised I would have more information on Final Fantasy XVI sometime later in quot producer Naoki Yoshida wrote on Twitter quot However I regret to inform you that I will be unable to keep that promise as complications from the ongoing Covid pandemic have delayed the game s development by almost half a year quot A message from Final Fantasy XVI producer Naoki Yoshida FF FFXVIpic twitter com qtfJUUpLAーFINAL FANTASY XVI finalfantasyxvi December Developer Square Enix allowed partners to work from home due to the COVID pandemic but that apparently affected the schedule quot This has unfortunately hampered communication from the Tokyo office which in turn has led to delays in ーor in extreme cases cancellations of ーasset deliveries from our outsource partners quot Yoshida said nbsp As we noted last year FF XVI was originally launched as a PlayStation exclusive and has a medieval setting rather than the more sci fi oriented environments of recent versions The company said that it established an quot all new development environment quot for the project and had to learn the ins and outs of the PS nbsp quot We have spent much of addressing this pandemic issue and hope to see its impact to a minimum by the new year allowing us to better focus on the tasks at hand increasing graphic resource quality refining combat mechanics fleshing out individual battles putting the finishing touches on cutscenes and conducting overall graphic optimization quot said Yoshida The plan is to now do a reveal in spring of and release the game some time after that nbsp	2021-12-27 14:21:40
海外TECH	The Apache Software Foundation Blog	The Apache Weekly News Round-up: week ending 24 December 2021	https://blogs.apache.org/foundation/entry/the-apache-weekly-news-round6	The Apache Weekly News Round up week ending December Happy Friday everyone The Apache community has had another great week Let s review what we ve been up to ASF Board nbsp management and oversight of the business affairs of the corporation in accordance with the Foundation s bylaws nbsp Next Board Meeting January Board calendar and minutes nbsp ASF Infrastructure nbsp our distributed team on three continents keeps the ASF s infrastructure running around the clock nbsp M weekly checks yield uptime at Performance checks across different service components spread over more than machines in data centers around the world View the ASF s Infrastructure Uptime site to see the most recent averages Apache Code Snapshot nbsp Over the past week Apache Committers changed lines of code over commits Top contributors in order are Gary Gregory Harikrishna Patnala Claus Ibsen Duo Zhang and Andi Huber Apache Project Announcements nbsp the latest updates by category Big Data nbsp Apache NiFi released nbsp Apache HBase alpha released nbsp Apache Parquet and released nbsp nbsp CVE Potential DoS in case of malicious Parquet file Build Management nbsp Apache Archiva released Content nbsp nbsp Apache JSPWiki released nbsp Apache Traffic Control released nbsp Apache Jackrabbit FileVault nbsp released nbsp Apache Tika and released Databases nbsp nbsp Apache Geode and released nbsp Data Management Platform nbsp Apache Ignite released IoT nbsp Apache PLCX released nbsp nbsp CVE Buffer overflow in PLCC via crafted server response nbsp Enterprise Processes Automation ERP nbsp Apache OFBiz released nbsp Libraries nbsp Apache Logj and released nbsp nbsp CVE Logj does not always protect from infinite recursion in lookup evaluation nbsp Apache MXNet Incubating released nbsp Apache Daffodil releasedMail nbsp Apache James released nbsp Messaging nbsp nbsp Apache Qpid JMS and released nbsp Apache Pulsar released nbsp Search nbsp Apache Lucene released nbsp Apache Solr released nbsp nbsp CVE Apache Solr information disclosure vulnerability through DataImportHandler nbsp Servers nbsp nbsp Apache HTTP Server released nbsp nbsp CVE Possible buffer overflow when parsing multipart content in mod lua nbsp nbsp CVE Possible NULL dereference or SSRF in forward proxy configurations nbsp Apache HttpComponents Core GA released Web Frameworks Apache Struts and released nbsp Workflow nbsp Apache DolphinScheduler released nbsp Apache Airflow released Did You Know nbsp Did you know that ASF Security posted the status of more than three dozen Apache Projects in relation to the recent Apache Logj vulnerability please check individual projects not included in this list for updates nbsp Did you know that Apache Roller which powers blogs apache org new v contains upgrades for more than a dozen dependencies including Logj along with many bug fixes and improvements to the code base nbsp Did you know that tax deductible donations support the ASF s day to day operations that benefit Apache Projects and their communities Donate online using ACH credit card PayPal Apple Pay Google Pay and Microsoft Pay Apache Community Notices nbsp The Apache Month in Review November nbsp nbsp and video highlights nbsp nbsp Watch quot Trillions and Trillions Served quot the documentary on the ASF nbsp full feature nbsp min quot Apache Everywhere quot min quot Why Apache quot min nbsp “Apache Innovation min nbsp nbsp ASF Annual Report FY nbsp Press release nbsp and nbsp Report nbsp PDF nbsp The Apache Way to nbsp Sustainable Open Source Success nbsp nbsp nbsp Foundation Reports and Statements nbsp Presentations from s ApacheCon Asia and ApacheCon Home are available on the nbsp ASF YouTube channel nbsp quot Success at Apache quot focuses on the people and processes behind why the ASF quot just works quot nbsp nbsp Inside Infra the new interview series with members of the ASF infrastructure team meet nbsp nbsp nbsp Chris Thistlethwaite nbsp nbsp nbsp Drew Foulks nbsp nbsp nbsp Greg Stein Part I nbsp nbsp nbsp nbsp Part II nbsp nbsp and Part III nbsp nbsp nbsp Daniel Gruno Part I nbsp nbsp and Part II nbsp nbsp nbsp nbsp Gavin McDonald Part I nbsp nbsp and Part II nbsp nbsp nbsp nbsp Andrew Wetmore Part I nbsp nbsp and Part II nbsp nbsp nbsp Chris Lambertus Part I nbsp nbsp nbsp and Part II nbsp nbsp Follow the ASF on social media nbsp TheASF on Twitter nbsp and nbsp The ASF page LinkedIn nbsp nbsp Follow the nbsp Apache Community on Facebook nbsp and nbsp Twitter nbsp nbsp Are your software solutions Powered by Apache nbsp Download amp use our quot Powered By quot logos Stay updated about The ASFFor real time updates sign up for Apache related news by sending mail to announce subscribe apache org and follow TheASF on Twitter For a broader spectrum from the Apache community nbsp nbsp provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers	2021-12-27 14:38:09
海外科学	NYT > Science	NASA-Russia Alliance Is Shaken by Events on Planet Earth	https://www.nytimes.com/2021/12/27/science/russia-nasa-spacex-asat.html	NASA Russia Alliance Is Shaken by Events on Planet EarthThe relationship between the nations space agencies is facing a series of difficult tests including an antisatellite weapon and friction over Ukraine	2021-12-27 14:37:35
金融	RSS FILE - 日本証券業協会	J-IRISS	https://www.jsda.or.jp/anshin/j-iriss/index.html	iriss	2021-12-27 15:06:00
ニュース	BBC News - Home	No new Covid rules to be announced in England on Monday	https://www.bbc.co.uk/news/uk-59803879?at_medium=RSS&at_campaign=KARANGA	covid	2021-12-27 14:31:29
ニュース	BBC News - Home	Shoppers shun Christmas sales as footfall drops amid Covid fears	https://www.bbc.co.uk/news/business-59802144?at_medium=RSS&at_campaign=KARANGA	london	2021-12-27 14:25:46
ニュース	BBC News - Home	Covid: Travel chaos spills into new week	https://www.bbc.co.uk/news/world-59798682?at_medium=RSS&at_campaign=KARANGA	festive	2021-12-27 14:06:52
ニュース	BBC News - Home	Surrey PC 'honestly shocked' to see crash car driven on M25	https://www.bbc.co.uk/news/uk-england-kent-59803301?at_medium=RSS&at_campaign=KARANGA	cobham	2021-12-27 14:47:59
北海道	北海道新聞	全日空、３年ぶり総合職採用再開　客室乗務員見送り	https://www.hokkaido-np.co.jp/article/628184/	全日本空輸	2021-12-27 23:18:00
北海道	北海道新聞	ミャンマー国軍　「日本支援村」を攻撃　全村民避難	https://www.hokkaido-np.co.jp/article/628136/	日本政府	2021-12-27 23:04:18
北海道	北海道新聞	石狩、札幌・東区で大雪　新千歳氷点下１９度　管内冷え込み続く	https://www.hokkaido-np.co.jp/article/628161/	冬型の気圧配置	2021-12-27 23:04:38
北海道	北海道新聞	ＮＹ円、１１４円後半　１カ月ぶり安値水準	https://www.hokkaido-np.co.jp/article/628177/	外国為替市場	2021-12-27 23:03:00
北海道	北海道新聞	待ちわびた成人式　晴れ着でにっこり	https://www.hokkaido-np.co.jp/article/628162/	新型コロナウイルス	2021-12-27 23:04:19
北海道	北海道新聞	札山の手　走り悔やむ　高校駅伝３７位　追い上げかなわず	https://www.hokkaido-np.co.jp/article/628164/	全国高校駅伝	2021-12-27 23:02:07