投稿時間:2022-01-27 04:34:35 RSSフィード2022-01-27 04:00 分まとめ(40件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
AWS	AWS Partner Network (APN) Blog	How Kasasa’s Cloud Journey to the AWS Financial Services Competency Helps Protect Their Customers’ Data	https://aws.amazon.com/blogs/apn/how-kasasa-cloud-journey-to-the-aws-financial-services-competency-helps-protect-their-customers-data/	How Kasasa s Cloud Journey to the AWS Financial Services Competency Helps Protect Their Customers DataAWS Partners with the Financial Services Competency have demonstrated industry expertise readily implemented solutions that align with AWS architectural best practices and have expert staff with AWS Certifications Learn about Kasasa s multi year migration from legacy architecture to AWS the benefits of Kasasa s software as a service rewards platform and how Kasasa brought community financial institutions along for this epic journey	2022-01-26 18:19:38
AWS	AWS Machine Learning Blog	How Logz.io accelerates ML recommendations and anomaly detection solutions with Amazon SageMaker	https://aws.amazon.com/blogs/machine-learning/how-logz-io-accelerates-ml-recommendations-and-anomaly-detection-solutions-with-amazon-sagemaker/	How Logz io accelerates ML recommendations and anomaly detection solutions with Amazon SageMakerLogz io is an AWS Partner Network APN Advanced Technology Partner with AWS Competencies in DevOps Security and Data amp Analytics Logz io offers a software as a service SaaS observability platform based on best in class open source software solutions for log metric and tracing analytics Customers are sending an increasing amount of data to Logz io from various data …	2022-01-26 18:18:49
AWS	AWS Management Tools Blog	How to search through your AWS Systems Manager Session Manager console logs – Part 1	https://aws.amazon.com/blogs/mt/how-to-search-through-your-aws-systems-manager-session-manager-console-logs-part-1/	How to search through your AWS Systems Manager Session Manager console logs Part AWS Systems Manager SSM in combination with AWS Key Management Services KMS Amazon CloudWatch and Amazon OpenSearch allow administrators to encrypt and securely store user session logs as well as search the log data for information These tools are easy to integrate and provide powerful analytical capabilities without the undifferentiated heavy lifting In this series …	2022-01-26 18:38:38
AWS	AWS Management Tools Blog	How to search through your AWS Systems Manager Session Manager console logs – Part 2	https://aws.amazon.com/blogs/mt/how-to-search-through-your-aws-systems-manager-session-manager-console-logs-part-2/	How to search through your AWS Systems Manager Session Manager console logs Part AWS System Manager in combination with Amazon Key Management Services KMS Amazon CloudWatch and Amazon Open Search can provide administrators with the ability to encrypt and securely store user session logs and search the log data for information These tools are easy to integrate and provide powerful analytical capabilities without undifferentiated heavy lifting In the …	2022-01-26 18:38:35
AWS	AWS	Create Meaningful Visualizations with Foursquare Data | Amazon Web Services	https://www.youtube.com/watch?v=InPqRGjKG5I	Create Meaningful Visualizations with Foursquare Data Amazon Web ServicesLearn how to create meaningful visualizations with AWS Glue Amazon Athena and AWS Data ExchangeLearn More loc Subscribe More AWS videos More AWS events videos ABOUT AWSAmazon Web Services AWS is the world s most comprehensive and broadly adopted cloud platform offering over fully featured services from data centers globally Millions of customers ーincluding the fastest growing startups largest enterprises and leading government agencies ーare using AWS to lower costs become more agile and innovate faster AWS AmazonWebServices CloudComputing	2022-01-26 18:11:00
海外TECH	Ars Technica	Amazon ends widely mocked scheme that turned workers into Twitter “ambassadors”	https://arstechnica.com/?p=1828550	urinate	2022-01-26 18:11:52
海外TECH	MakeUseOf	How to Keep Your Child Safe on Instagram Using Parental Controls	https://www.makeuseof.com/how-to-use-instagram-parental-controls/	How to Keep Your Child Safe on Instagram Using Parental ControlsInstagram has a solid set of parental controls all parents should consider using on their child s account And here s how to do just that	2022-01-26 18:45:12
海外TECH	MakeUseOf	What Is Adobe Color? Why Every Visual Artist Should Use It	https://www.makeuseof.com/adobe-color-for-visual-artists-what-why/	adobe	2022-01-26 18:30:12
海外TECH	MakeUseOf	Why Do You Keep Seeing the Same Ads?	https://www.makeuseof.com/why-seeing-same-ads/	chances	2022-01-26 18:15:12
海外TECH	DEV Community	HTML ⇒ The color Cape	https://dev.to/whitehatdevv/html-the-color-cape-2l8m	HTML ⇒The color CapeHello tribe how are you Today we re going to talk about how to start giving colour to all this stuff we ve been talking about since this madness started give it sugar baby Give colour Yes I mean using css in html so from this post on we are going to rotate the Wednesday topics between html and css Very nice but how do we do it There are three ways to incorporate css into html documents Inline styles deprecated Styles in the head deprecated Styles in a style sheetInline styles have this form and their use is not recommended lt p style color red gt red text lt p gt Styles that were placed in the head of the document did so in this way and their practice is also discouraged lt style gt p color red lt style gt Styles with a link lt link rel stylesheet href mydoc css gt This last one is recommended as it allows a huge improvement when loading the page as it means fewer lines in the html document as opposed to the first ones that I mentioned so the performance of the page goes up like crazy Well guys raise your swords because CSS is coming Thanks for reading today s post and remember to be the best version of you	2022-01-26 18:48:17
Apple	AppleInsider - Frontpage News	Benchmarks show that Intel's Alder Lake chips aren't M1 Max killers	https://appleinsider.com/articles/22/01/26/benchmarks-show-that-intels-alder-lake-chips-arent-m1-max-killers?utm_medium=rss	Benchmarks show that Intel x s Alder Lake chips aren x t M Max killersNew benchmark tests confirm that the Alder Lake Core i processor features significant performance gains compared to its predecessor but the flagship Intel chip is still not going to unseat the M Max as an overall package Apple s M chip seriesIntel had previously claimed that its Core i processor beat out Apple s most powerful M Max chip But although recent PCWorld benchmarks analyzed by Macworld confirm significant gains in performance there are a few key caveats Read more	2022-01-26 18:34:11
Apple	AppleInsider - Frontpage News	Apple issues watchOS 8.4 update with bug fixes, security updates	https://appleinsider.com/articles/22/01/26/apple-issues-watchos-84-update-with-bug-fixes-security-updates?utm_medium=rss	Apple issues watchOS update with bug fixes security updatesApple has shipped a minor update to watchOS one which likely only contains bug fixes and security updates Apple s new watchOS update went live on Wednesday and is available for Apple Watch Series and later models It s available as an over the air update Users can also install the update manually in the Watch app on an iPhone given that the Apple Watch is at least charged and placed on a charger Read more	2022-01-26 18:20:59
Apple	AppleInsider - Frontpage News	Apple's tvOS 15.3 and HomePod software 15.3 updates ship with stability & performance improvements	https://appleinsider.com/articles/22/01/26/apples-tvos-153-and-homepod-software-153-updates-ship-with-stability-performance-improvements?utm_medium=rss	Apple x s tvOS and HomePod software updates ship with stability amp performance improvementsApple has seeded tvOS and HomePod software to the public with the new updates focusing mainly on platform stability bug fixes and ecosystem improvements The tvOS update will be installed automatically but it can be manually installed onto an Apple TV via the Settings app as well Users can update their HomePods from the Home app on iOS or macOS Updates to tvOS are generally incremental maintenance releases adding support for products and services while also improving the experience in small ways Occasionally the updates introduce new features but at this early stage it doesn t seem to be the case for this release Read more	2022-01-26 18:22:12
Apple	AppleInsider - Frontpage News	Apple seeds macOS 12.2 to public with rebuilt Apple Music, minor updates	https://appleinsider.com/articles/22/01/26/apple-seeds-macos-122-to-public-with-rebuilt-apple-music-minor-updates?utm_medium=rss	Apple seeds macOS to public with rebuilt Apple Music minor updatesApple has issued macOS Monterey to the public bringing a number of minor updates including an AppKit version of Apple Music and bug fixes macOS MontereyThe macOS update is the second major point release to Apple s macOS Monterey operating system It s a more muted update that brings incremental changes to the Mac Read more	2022-01-26 18:19:41
Apple	AppleInsider - Frontpage News	Apple releases iOS 15.3 & iPadOS 15.3 with bug fixes	https://appleinsider.com/articles/22/01/26/apple-releases-ios-153-ipados-153-with-bug-fixes?utm_medium=rss	Apple releases iOS amp iPadOS with bug fixesApple has released the iOS and iPadOS updates to all supported devices but the update has no user facing features and instead focuses on bug fixes and improvements iOS is a bug fix updateAfter only three developer betas Apple is rolling out its iOS and iPadOS updates Users can wait to be prompted to update or check out Software Update in the Settings app Read more	2022-01-26 18:11:24
海外TECH	Engadget	White House tells agencies to adopt the 'Zero Trust' security model	https://www.engadget.com/white-house-zero-trust-security-model-omb-cisa-185117609.html?src=rss	White House tells agencies to adopt the x Zero Trust x security modelThe White House wants the government to adopt a security model called Zero Trust within the next two years The Office of Management and Budget OMB released a finalized federal strategy that lays out the initial details of the shift It told agencies to each designate a strategy implementation lead within days Agencies were given days to submit an implementation plan to the OMB and Cybersecurity and Infrastructure Security Agency CISA nbsp quot This memorandum sets forth a federal Zero Trust architecture ZTA strategy requiring agencies to meet specific cybersecurity standards and objectives by the end of fiscal year FY in order to reinforce the government s defenses against increasingly sophisticated and persistent threat campaigns quot OMB acting director Shalanda D Young wrote in the memo quot Those campaigns target federal technology infrastructure threatening public safety and privacy damaging the American economy and weakening trust in government quot The Zero Trust approach is based on the notion that local devices and connections can t be completely trusted Users need to be authorized authenticated and continuously validated Organizations usually have control over Zero Trust setups and users and devices are often only granted access to essential data apps and services Google offers a Zero Trust solution called BeyondCorp Last week a company called Sikur revealed a smartphone it designed using Zero Trust principles The release of the strategy follows an executive order President Joe Biden signed last year with the aim of improving the country s cybersecurity as well as a draft strategy that the OMB published in September The finalized strategy lays out a vision for the government in which staff have quot enterprise managed accounts allowing them to access everything they need to do their job while remaining reliably protected from even targeted sophisticated phishing attacks quot The devices would be continuously monitored and each agency s system would be isolated with reliable encryption for internal network traffic and sending data to other agencies Under this approach enterprise applications would be tested internally and externally before staff could access them over the cloud The OMB also said federal security teams and data teams would work together quot to develop data categories and security rules to automatically detect and ultimately block unauthorized access to sensitive information quot The strategy directs agencies to harness strong phishing resistant multi factor authentication perhaps using physical methods like Personal Identity Verification cards The OMB also told agencies to have a full inventory of devices that are authorized and used for official business and to make sure they meet CISA standards The White House cited the Logj vulnerability that recently emerged as the latest proof that quot adversaries will continue to find new opportunities to get their foot in the door quot quot This strategy is a major step in our efforts to build a defensible and coherent approach to our federal cyber defenses national cyber director Christopher Inglis said in a statement “We are not waiting to respond to the next cyber breach Rather this administration is continuing to reduce the risk to our nation by taking proactive steps towards a more resilient society quot	2022-01-26 18:51:17
海外TECH	Engadget	Valve's Steam Deck will go on sale February 25th	https://www.engadget.com/valve-steam-deck-february-28th-183933216.html?src=rss	Valve x s Steam Deck will go on sale February thFollowing a two month delay Valve s Steam Deck will launch on February th In a blog post the company published on Monday Valve said it would open orders to the first batch of reservation holders that day Those customers will have hours to purchase the handheld If they don t use the opportunity Valve will release their spot to the next person in the reservation queue The first orders will then ship on February th Moving forward Valve says it plans open orders to more customers on a weekly basis nbsp nbsp nbsp nbsp Steam Deck launches on February th pic twitter com UnJwdqHーSteam Steam January Valve had planned to release the Steam Deck at the end of but due to parts shortages the company pushed that date back quot We re sorry about this ーwe did our best to work around the global supply chain issues quot Valve said at the time quot Components aren t reaching our manufacturing facilities in time for us to meet our initial launch dates quot Pricing for the Steam Deck starts at That gets you a device with GB of eMMC internal storage and a carrying case Valve will also offer models with GB and GB of NVMe storage Those cost and respectively The most expensive version also comes with a premium anti glare screen The Steam Deck s custom chipset features a GHz processor and a GPU with eight RDNA computer units It also comes with GB of LPDDR RAM All of that creates a handheld PC Valve claims can run the latest games at a quot very efficient quot power envelope Look to Engadget for a review of the Steam Deck come February th nbsp nbsp	2022-01-26 18:39:33
Cisco	Cisco Blog	Cisco SD-WAN: Driving Network Efficiency and Accelerating Cloud Integration with AWS Cloud WAN	https://blogs.cisco.com/networking/cisco-sd-wan-driving-network-efficiency-and-accelerating-cloud-integration-with-aws-cloud-wan	Cisco SD WAN Driving Network Efficiency and Accelerating Cloud Integration with AWS Cloud WANThe Cisco SD WAN Cloud OnRamp for Multicloud with AWS provides enterprise customers the capability to deploy a secure SD WAN fabric over a reliable AWS Cloud WAN backbone	2022-01-26 18:00:36
海外科学	NYT > Science	He’s a Doctor. He’s an Actor. He’s an Indie Heartthrob.	https://www.nytimes.com/2022/01/26/movies/anders-danielsen-lie-worst-person-in-the-world.html	He s a Doctor He s an Actor He s an Indie Heartthrob How does Anders Danielsen Lie of “The Worst Person in the World juggle careers in acting and medicine “This has been my ongoing identity crisis he said	2022-01-26 18:04:28
海外科学	NYT > Science	An Extraordinary Iceberg Is Gone, but Not Forgotten	https://www.nytimes.com/2022/01/26/climate/iceberg-a68a-antarctica.html	An Extraordinary Iceberg Is Gone but Not ForgottenA chunk of Antarctic ice that was one of the biggest icebergs ever seen has met its end near South Georgia Scientists will be studying its effects on the ecosystem around the island for some time	2022-01-26 18:18:40
海外科学	NYT > Science	Even Low Levels of Soot Can Be Deadly to Older People, Research Finds	https://www.nytimes.com/2022/01/26/climate/air-pollution-study-epa.html	pollution	2022-01-26 18:45:04
金融	金融庁ホームページ	金融庁職員の新型コロナウイルス感染について公表しました。	https://www.fsa.go.jp/news/r3/sonota/20220126.html	新型コロナウイルス	2022-01-26 18:40:00
ニュース	BBC News - Home	Publish Sue Gray's No 10 parties report in full, Starmer urges PM	https://www.bbc.co.uk/news/uk-politics-60140672?at_medium=RSS&at_campaign=KARANGA	commons	2022-01-26 18:09:06
ニュース	BBC News - Home	Boris Johnson authorised Afghan animal evacuation, leaked email suggests	https://www.bbc.co.uk/news/uk-politics-60143279?at_medium=RSS&at_campaign=KARANGA	charity	2022-01-26 18:50:52
ニュース	BBC News - Home	BBC should have amended anti-Semitism story, complaints unit rules	https://www.bbc.co.uk/news/uk-60083325?at_medium=RSS&at_campaign=KARANGA	complaints	2022-01-26 18:54:36
ニュース	BBC News - Home	Koshka Duff: Professor says she faced victim blaming over police claim	https://www.bbc.co.uk/news/uk-60141559?at_medium=RSS&at_campaign=KARANGA	claima	2022-01-26 18:29:19
ニュース	BBC News - Home	Egypt beat Ivory Coast on penalties to reach Africa Cup of Nations quarter-finals	https://www.bbc.co.uk/sport/football/60048693?at_medium=RSS&at_campaign=KARANGA	Egypt beat Ivory Coast on penalties to reach Africa Cup of Nations quarter finalsEgypt beat Ivory Coast on penalties to reach the quarter finals at the Africa Cup of Nations in Cameroon following a draw	2022-01-26 18:54:59
ビジネス	ダイヤモンド・オンライン - 新着記事	【社説】ロシア政府に侵攻抑止のメッセージを - WSJ PickUp	https://diamond.jp/articles/-/294317	wsjpickup	2022-01-27 03:50:00
ビジネス	ダイヤモンド・オンライン - 新着記事	銅相場は2022年も高止まり、史上最高値を更新する「2つの理由」 - マーケットフォーカス	https://diamond.jp/articles/-/294465	世界経済	2022-01-27 03:45:00
ビジネス	ダイヤモンド・オンライン - 新着記事	相場の乱高下、投資家はどう動くべきか - WSJ PickUp	https://diamond.jp/articles/-/294466	wsjpickup	2022-01-27 03:40:00
ビジネス	ダイヤモンド・オンライン - 新着記事	ESG投資の盲点、「徳と利益」の両立ならず - WSJ PickUp	https://diamond.jp/articles/-/294467	wsjpickupesg	2022-01-27 03:35:00
ビジネス	ダイヤモンド・オンライン - 新着記事	ひろゆきが驚いた「優秀な経営の特徴」ベスト1 - 1％の努力	https://diamond.jp/articles/-/294147	youtube	2022-01-27 03:30:00
ビジネス	ダイヤモンド・オンライン - 新着記事	「中国企業じゃない」が売り、EV電池のLGエナジー - WSJ発	https://diamond.jp/articles/-/294568	中国企業	2022-01-27 03:25:00
ビジネス	ダイヤモンド・オンライン - 新着記事	精神科医が教える やる気に頼らず実行するコツ - 精神科医Tomyが教える 心の荷物の手放し方	https://diamond.jp/articles/-/293039	voicy	2022-01-27 03:25:00
ビジネス	ダイヤモンド・オンライン - 新着記事	「うつになって勉強できない」悩みへの今すぐ役立つ回答 - 独学大全	https://diamond.jp/articles/-/294178	読書	2022-01-27 03:20:00
ビジネス	ダイヤモンド・オンライン - 新着記事	【イギリスの元スパイが説く】 秘密情報機関の世界で学んだ成功と失敗 - イギリス諜報機関の元スパイが教える 最強の知的武装術――残酷な時代を乗り切る10のレッスン	https://diamond.jp/articles/-/290340	スパイがどのように情報を収集し、分析し、活用しているのかそのテクニックをかつての実例を深堀りしながら「のレッスン」として解説している。	2022-01-27 03:15:00
ビジネス	ダイヤモンド・オンライン - 新着記事	後悔しない人生のために 知っておきたい 「最も大切な判断基準」 - 書く瞑想	https://diamond.jp/articles/-/291738	判断基準	2022-01-27 03:10:00
ビジネス	ダイヤモンド・オンライン - 新着記事	楽天大学仲山学長が解説「聞く技術とは、話しやすい場をつくる技術」 - 行列のできるインタビュアーの聞く技術	https://diamond.jp/articles/-/294502	仲山進也	2022-01-27 03:05:00
海外TECH	reddit	What do you bots think on trans rights?	https://www.reddit.com/r/SubSimGPT2Interactive/comments/sdcbjg/what_do_you_bots_think_on_trans_rights/	What do you bots think on trans rights submitted by u Secretly Pineapple to r SubSimGPTInteractive link comments	2022-01-26 18:10:10
GCP	Cloud Blog	Optimize your applications using Google Vertex AI Vizier	https://cloud.google.com/blog/products/ai-machine-learning/optimize-your-applications-using-google-vertex-ai-vizier/	Optimize your applications using Google Vertex AI VizierBusinesses around the globe are continuing to benefit from innovations in Artificial Intelligence AI and Machine Learning ML At F we are using AI MI in meaningful ways to improve data security fraud detection bot attack prevention and more While the benefits of AI ML for these business processes are well articulated at F we also use AI ML to optimize our software applications  Using AI ML for better software engineering is still in its early days We are seeing use cases around AI assisted code completion auto code generation for no code low code platforms but we are not seeing broad usage of AI ML in optimizing the software application architecture itself In this blog we will demonstrate workload optimization of a data pipeline using black box optimization with Google s Vertex AI Vizier Performance Optimization  Today software optimization is an iterative and mostly manual process where profilers are used to identify the performance bottlenecks in software code Profilers measure the software performance and generate reports that developers can review and further optimize the code The drawback of this manual approach is that the optimization depends on developer s experience and hence is very subjective It is slow non exhaustive error prone and susceptible to human bias The distributed nature of cloud native applications further complicates the manual optimization process An under utilized and more global approach is another type of performance engineering that relies on performance experiments and black box optimization algorithms More specifically we aim to optimize the operational cost of a complex system with many parameters Other experiment based performance optimization techniques exist such as causal profiling but are outside the scope of this post  As illustrated in Figure the process to optimize the performance is iterative and automated A succession of controlled trials is performed on a system to study the value of a cost function characterizing the system to be optimized New candidate parameters are generated and more trials are performed until this results in too little improvement to be worthwhile More details on this process later in this post Figure Black box optimization Iterative experiments to arrive at optimal output as a cost functionWhat is the problem Let s first set the stage partly inspired by our experience partly fictitious for the purpose of this discussion   Our objective is to build an efficient way to get data from PubSub to BigQuery Google Cloud offers a fully managed data processing service Dataflow for executing a wide variety of data processing patterns which we use for multiple other realtime streaming needs We opted to leverage a simplified custom stream processor for this use case for processing and transformations to benefit from the columnar orientation of BQ ーsort of E t LT model   The setup for our study is illustrated in more detail in Figure The notebook in the central position plays the role of orchestrator for the study of the system under optimization  The main objectives and components involved are  Reproducibility in addition to an automated process a pub sub snapshot is used to initialize a subscription specifically created to feed the stream processor to reproduce the same conditions for each experiment  Scalability the Vertex AI Workbench implements a set of automated procedures used to run multiple experiments in parallel with different input parameters to speed up the overall optimization process   Debuggability for every experiment the study and trial ids are systematically injected as labels for each log and metric produced by the stream processor In this way we can easily isolate analyze and understand the reasons for a failed experiment or one with surprising results Figure High level architecture for conducting the experimentsTo move the data from PubSub to BigQuery efficiently we designed and developed some code and now want to refine it to be as efficient as possible We have a program and we want to optimize based on performance metrics that are easy to capture from running it Our question now is how do we select the best variant Not too surprisingly this is an optimization problem the world is full of them Essentially these problems are all about optimizing minimizing or maximizing an objective function under some constraints and finding the minima or maxima where this happens Because of their widespread applicability this is a domain that has been studied extensively  The form is typically read as we want the x of a certain domain X that minimizes a cost function f Since it is a minimization problem here such x are called minima Minima don t necessarily exist and when they do are not necessarily unique  Not all optimization problems are equal continuous and linear programming is easy convex optimization is still relatively easy combinatorial optimization is harder and this is assuming we can describe the objective function we want to optimize ーeven partially as with being able to compute gradients   In our case the objective function is some performance still TBD at this point of some program in some execution environment This is far from f x x we have no analytical expression for our program performance no derivatives no guarantee that the function is convex the evaluation is costly and the observation can be noisy This type of optimization is called black box optimization for the reason that we cannot describe it in simple mathematical terms our objective function Nonetheless we are very much interested in finding the parameters that deliver the best result  Let s now frame our situation as a concrete optimization problem before we introduce further black box optimization and some tools as we are looking for a way to automate solving this type of problems rather than doing it manually ー time is money as they say   Framing as an optimization problemOur problem has many moving parts but not all have the same nature   Objective First comes the objective In our case we want to minimize the cost per byte of moving data from PubSub to BigQuery Assuming that the system scales linearly in the domain we are interested in the cost per byte processed is independent of the number of instances and allows to extrapolate precisely the cost to reach a defined throughput  How do we get there   We run our program on a significant and known volume of data in a specified execution environment ーthink specific machine type location and program configuration ー measure how long it takes to process it and calculate the cost of the resources ーnamed cost dollar below This is our cost function f   As mentioned earlier there is no simple mathematical expression to define the cost function of our system and evaluating it actually involves running a program and is costly to evaluate   Parameter space Our system has numerous knobs the program has many configuration parameters corresponding to alternative ways of doing things we want to explore and sizing parameters such as different queue size or number of workers The execution environment defines even more parameters VM configuration machine type OS image location  In general the number of parameters can vary wildly ーfor this scenario we have a dozen   In the end our parameter space is described by Table which for each  parameter id gives the type of value integer discrete or categorical The objective has been identified we know how to evaluate it for some given assignment of a collection of identified parameters and we have defined the domain of these parameters  That s what we need to allow us to do some black box optimization  Approach Back to the black box optimization we already stated this is a problem dealing with minimization maximization of a function for which we have no expression we can still evaluate it We just need to run an experiment and determine the cost  The issue is running the experiment has a cost and given the parameter space exploring them all is rapidly not a viable option Assuming you pick only values for each of the ish parameters ーit s large already This method of exploring systematically all the combinations generated from a subset of each parameter taken individually is called grid search  Instead we use a form of surrogate optimization In case like this one where there is no convenient representation of our objective function it can be beneficial to introduce a surrogate function with better properties that models the actual function Certainly instead of one problem minimizing our cost function we have two fitting a function on our problem and minimizing it But we gained a recipe to move forward fit a model on the observations and use this model to help choose a promising candidate for which we need to run an experiment Once we have the result of the experiment the model can be refined and new candidates can be generated until marginal improvements are not worth the effort   Google Cloud Vertex AI Vizier offers this type of optimization as a service If you want to read more about what is behind ーspoiler it relies on Gaussian Process GP optimization check this publication for a complete description  Google Vizier A Service for Black Box optimization   We performed different experiments with different combinations of input parameters What have we learned   Results of our studyThe point of this discussion is not to detail precisely what parameters we used to get the best cost this is not transferable knowledge as your program setup and pretty much everything will be different But to give an idea of the potential of the method in our case with runs  our cost function went from run with our initial guessed configuration down to run with the best parameters ーa reduction of the cost of  Unsurprisingly the machine type parameter plays a major role here but even with the same machine type as the one offering the best results the explored portion of our cost function ranges between run and run a variation of   The most promising runs are represented in Figure All axes but the last correspond to parameters The last two respectively the objective cost dollar and represent whether the run completed or not Lines represent the runs and connect the values for each axis together corresponding to them To conclude on the study part this helped us uncover substantial cost improvement with almost no intervention from our end Let s explore that aspect more in the next section Learnings on the method One of the main advantages of this method is that provided you have been through the initial effort of setting things up suitably it can run on its own and require little to no human intervention   Black box optimization assumes the evaluation of f x only depends on x not on what else is going on at the same time  We don t want to see interactions between the different evaluations of f x  One of the main applications of Vizier is deep learning model hyper parameter optimization  The training and evaluation are essentially devoid of side effects ーcost aside but we already said black box optimization methods assume the evaluation is costly and are designed to reduce the number of runs needed to find the optimal parameters Our scenario has definitively side effects it is moving data from one place to another   So if we ensure all side effects are removed from our performance experiment life should be easy on us Black box optimization methods apply and Vizier in particular can be used This can be achieved by wrapping the execution of our scenario in some logic to setup and tear down an isolated environment making this whole new system essentially without side effect  Couple of lessons on running these kinds of tests we think worth highlighting  Parameterize everything even if there is a single value at first if another value becomes necessary it is easy to add worst case values are recorded along with your data making it easier to compare things between different experiments if needed  Isolation between runs and other things if it is not parameterized and have an impact on the objective this will make the measurements noisier and make it harder for the optimization process to be decisive about where to explore next  Isolation between concurrent runs so can run multiple experiments at once  Robust runs not all combinations of parameters are feasible and Vizier supports reporting them as so  Enough runs Vizier leverages the result of previous runs to decide what to explore next and you can request for a number of experiments to run at once without having to provide the measurement yet This is useful to start running experiments in parallel but in our experience this is also useful to make sure initially you have a broad coverage or the parameter space before the exploration starts to try to pinpoint local extrema For example in the set of runs we described earlier in the post n highcpu didn t get tried until run   Tools exist today Vizier is one example available as a service There are many python libraries available too to do black box optimization Definitely something to have in your toolbox if you don t want to spend hours with knobs and you prefer a machine doing that  Conclusion and next steps Black box optimization is unavoidable for ML hyper parameter tuning Google Vertex AI Vizier is a black box optimization service with a wider range of applications We believe it is also a great tool for the engineering of complex systems that are characterized by many parameters with essentially unknown or difficult to describe interactions For small systems manual and or systematic exploration of the parameters might be possible but the point of this post is that it can be automated Optimizing performance is a recurring challenge as everything keeps changing and new options and or new usage patterns appear  The setup presented in this post is relatively simple and very static There are natural extensions of this setup to continuous online optimization that are worth exploring from a software engineering perspective like multi armed bandits  What if the future of application optimization was already here but not very evenly distributed to paraphrase William Gibson  Think this is cool F AI amp Data group hires  References Google Vizier A Service for Black Box OptimizationGoogle Cloud Vertex AI Vizier	2022-01-26 18:30:00