投稿時間:2022-02-08 21:25:01 RSSフィード2022-02-08 21:00 分まとめ(27件)

カテゴリー等	サイト名等	記事タイトル・トレンドワード等	リンクURL	頻出ワード・要約等/検索ボリューム	登録日
IT	ITmedia 総合記事一覧	[ITmedia ビジネスオンライン] 人気の駅ランキング東京23区編　3位「新小岩」、2位「三軒茶屋」、1位は？	https://www.itmedia.co.jp/business/articles/2202/08/news161.html	itmedia	2022-02-08 20:08:00
python	Pythonタグが付けられた新着投稿 - Qiita	Django models to DataFrame to csv	https://qiita.com/sphink/items/10550492d0347a57a513	DjangomodelstoDataFrametocsvfromdatetimeimportdatetimeimportnumpyfromdjangocoremanagementimportBaseCommandfromdjangodbmodelsfieldsimportDateTimeFieldBooleanFieldfrommodelimportYourModelimportpandasaspdUsageExportModelYourModelForeignKeyclassExportModeldefinitselfmodelkeyselftargetmodelmodelselfkeykeyselftargetkeyvaluekeykeydefexecselfifselftargettablecheckexistskeyselftargetmodelexportdatadeftargettablecheckexistskeyselfselftargetmodelfilterdkeyqsselftargetmodelobjectsfilterselftargetkeyvalueiflenselftargetmodelfilterdkeyqsgtreturnTrueelsereturnFalsedeftargetmodelexportdataselfdfpdDataFramefromrecordsselftargetmodelfilterdkeyqsvaluesforcolinselftargetmodelmetafieldsifcolclassDateTimeFielddfcolnamepdtodatetimedfcolnamedfcolnamedfcolnamedtstrftimeYmdHMSifcolclassBooleanFielddfcolnamedfcolnamereplaceFalsedfcolnamedfcolnamereplaceTrueほとんどのFiledsのNoneはNAで抽出できるdfdfreplacepdNANULLDateTimeFieldsのNullはNaNdfdfreplacenumpyNaNNULLcsvfilenamefselfkeyselftargetmodelnamedeleteddatetimenowstrftimeYmdHMcsvdftocsvcsvfilenameindexFalse概要DjangoのモデルからDataFrameに変換して、Sequelとかにそのまま読み込ませれるような形式でCSVに出力するサンプルコードです。	2022-02-08 20:11:12
js	JavaScriptタグが付けられた新着投稿 - Qiita	たった10行で簡単に絞込みセレクトボックス（Javascript無し）	https://qiita.com/alien/items/c588e9718fcfeb94bdcd	実際には、空配列かdisabledにするのでしょうが、例なので、初期状態では全てのデータが見られるようにしています。	2022-02-08 20:09:22
Ruby	Rubyタグが付けられた新着投稿 - Qiita	たった10行で簡単に絞込みセレクトボックス（Javascript無し）	https://qiita.com/alien/items/c588e9718fcfeb94bdcd	実際には、空配列かdisabledにするのでしょうが、例なので、初期状態では全てのデータが見られるようにしています。	2022-02-08 20:09:22
Ruby	Railsタグが付けられた新着投稿 - Qiita	たった10行で簡単に絞込みセレクトボックス（Javascript無し）	https://qiita.com/alien/items/c588e9718fcfeb94bdcd	実際には、空配列かdisabledにするのでしょうが、例なので、初期状態では全てのデータが見られるようにしています。	2022-02-08 20:09:22
海外TECH	MakeUseOf	6 Reasons Why You Should NOT Buy a Bluetooth Keyboard	https://www.makeuseof.com/tag/6-reasons-not-buy-bluetooth-keyboard/	bluetooth	2022-02-08 11:55:44
海外TECH	MakeUseOf	How to Spot a Bad Privacy Policy?	https://www.makeuseof.com/spot-bad-privacy-policy/	collection	2022-02-08 11:30:22
海外TECH	MakeUseOf	What Is Flickr? A Beginner's Guide to the Photo-Sharing Platform	https://www.makeuseof.com/what-is-flickr-guide-photo-sharing/	What Is Flickr A Beginner x s Guide to the Photo Sharing PlatformPhotographers prefer uploading their work to Flickr rather than Instagram But what exactly is Flickr And what is Flickr used for Let s dive in	2022-02-08 11:15:42
海外TECH	DEV Community	Jenkins Pipeline Tutorial | Jenkins Tutorial | Part VIII	https://dev.to/lambdatest/jenkins-pipeline-tutorial-jenkins-tutorial-part-viii-1f6	Jenkins Pipeline Tutorial Jenkins Tutorial Part VIIIThis video demonstrates the benefits of Jenkins CI CD pipeline installing Pipeline Maven Integration Plugin with an example of how to create a Jenkins pipeline The video is a part of the Jenkins Tutorial series for beginners In this video Moss tech with moss explains steps to create a Jenkins CI CD Pipeline using the Jenkins UI Learn how to use a Maven project as the codebase to be built in the pipeline amp also use the Pipeline Maven plugin to facilitate the use of Maven within the Pipeline script You will get to learn What are the benefits of Jenkins Pipeline What are the different types of Jenkins Pipeline What is Pipeline job in Jenkins How does a Jenkins Pipeline work	2022-02-08 11:51:12
海外TECH	DEV Community	Data Science Workflows — Notebook to Production	https://dev.to/barazida/data-science-workflows-notebook-to-production-13i2	Data Science Workflows ーNotebook to ProductionReduce the time from research to production using various MLOps tools and techniques Refactoring a project developed in a notebook to a production ready state can be challenging and consume our time and resources At DagsHub we ve interviewed data science practitioners and ML engineers from hundreds of companies trying to get to the bottom of their workflow problems and solutions In this blog I will construct our two years of research into a structured workflow that will help YOU reduce the iteration time between research and production using MLOps tools and techniques Considerations When Deploying a Model to ProductionImplementing DagsHub s product playbook I d like to start our journey at the end zone by thinking about our considerations when deploying a model to production When I think about production the first thing that comes to mind is delivering a product to a different team ーin or outside the company Therefore we d like our results to be reliable and in many cases explainable We d also like to reduce the risk of bugs in production therefore our project should be testable not only code but also model data pipeline etc When a bug arises in production and it will we need a way to reproduce the production s state This means not only having access to all different components but also the relevant versions Last we d like the deploying process to be as automatic as possible reducing the friction between the research team and the MLOps team Based on those considerations let s go back to the starting point We ll usually start a project by testing and prototyping our hypothesis and our go to tool will be Notebooks Project JupyterBorn out of IPython in Project Jupyter has seen enthusiastic adoption among the data science community to become the de facto standard tool for prototyping Its impact was officially acknowledged in when chosen to receive the ACM Software Systems Award an honor shared with Unix the Web Java and such However if you explore some of the data science communities you d get the impression that notebooks are the root of all evil I disagree I think that Notebooks are awesome They give us an interactive way to work with our code Many companies DagsHub included choose the notebook interface to host their tutorials and interactive manuals They have built in visualizations a benefit for communicating your work with technical colleagues or with not so technical stakeholders All of this means they are great for quick prototyping The Downside of Using Notebooks in ProductionHowever when your project evolves and grows in complexity not to mention moving to production you most likely hit walls Based on our research we mapped five core difficulties data scientists face Reproducibility ーWith notebooks we can run out of order code and edit the cells after they have already been run once A huge disadvantage for reproducibility which makes us put a lot of effort into tracking the kernel state when executing an experiment Version Control ーJupyter notebooks are basically large JSON files that can t easily be diffed by Git which makes it hard to review not to mention merge changes Debug ーNotebooks have a debugger tool but as the notebook gets longer it becomes a real pain to use Testing amp Reusing ーThe code hosted in notebook cells is not callable from external locations and can t be tested easily CI CD ーLacking proper CI CD tools to automate the deployment process If you encountered other challenges while working with Notebooks ーI d love to hear about them Some even go as far as deploying the notebook to production which in my opinion is not recommended It disregards the industry standards workflows and best practices Also we re still missing proper CI CD tools to support the deployment lifecycle the notebook requires significant cleanup and packaging of libraries outside of the data scientist skillset and it s hard to run while using other tools in production “But hi Netflix deploys their notebook to production an argument used a lot in the MLOps community Well yes some great companies use notebooks in production but they invest a lot of resources to support such workflows which in most cases you ll not have or want to deflect for this task Components of a SolutionHaving the above challenges and production considerations in mind I gathered the best practices currently used in the industry into a structured workflow based on six components Convert to Scripts ーconvert the code into functions hosted in scripts Monorepo Strategy ーUse the “monorepo for data science structure to reduce complexity and scale our work Version EVERYTHING ーversion all of the project components ーcode data model etc Experiments Tracking ーautomate the logging process of experiments to aid our understanding of the project history and enable people to review the high level results Export Logic ーmove the logic steps to external scripts to avoid using notebooks in production and maintain one codebase Unit Testing ーWrite tests for the project components and invest more efforts in testing components that are low level and form the basis for other components Example scenarioIn our example scenario the team lead just got off the phone with the board and there is a new project in the funnel ー“Save The World They just sent the dataset and obviously ーthey want a model in production ASAP So we open a new notebook and get our feet wet Convert to ScriptsOur initial step will probably be exploring the data at hand For example check for feature distribution Now comes the most important conceptual change we need to make After getting an initial piece of code to do what It was designed to do we refactor it into a python function that doesn t rely on any global variables and takes everything as arguments We will store the function in python script outside the notebook which will make it callable from anywhere within the project Next we will import the function to our notebook and use it Capabilities UnlockedCode versioning ーVersion control our code using standard tools No hidden state ーThe function s inputs and outputs are clearly defined with no side effects The code is cleaner testable and repeatable Stay DRY don t repeat yourself ーReuse the code in this project or anywhere else Lint ーUse static code analysis tools an industry standard to flag programming errors bugs stylistic errors and suspicious constructs Monorepo strategyNow that we made our project much more modular we d like to leverage it and become more scalable by implementing the good old monorepo strategy Monorepo “mono meaning single and “repo is short for repository is a trunk based development implementation where code for many projects is stored in the same repository Well known companies use this strategy such as Google Facebook Microsoft and more What has monorepo got to do with Data Science Data science projects can be divided into stages that are completely independent of the other e g EDA data processing modeling etc If we address each step as a separate component we will be able to divide the project into sub tasks that are actually sub projects Using this mindset we ll be able to scale our work by dividing the tasks between different team members that will be independent of one another and conquer the project simultaneously We will notice that each stage depends only on the output of the previous one For example the modeling step only needs the output of the processing stage This is why we d like to open the bottleneck by producing a minimum valuable pipeline MVP and have each sub team start working on its task Once a team reaches a useful product they release an update of their output Now we ll have a python module for every sub task that stands on its own and as an integral component in the project s pipeline Moreover Instead of having one long notebook for the entire project we ll have a notebook per task Capabilities UnlockedScale ーWe can scale the number of collaborators on the project increase our product time and move X times faster to production Maintain review amp debug ーHaving shorter notebooks that are task oriented will make it easier to maintain review amp debug them Onboarding ーThe clear structure of the project makes the onboarding process easier Version ControlOnce reaching a point where we re collaborating with other people it rings only one bell ーVersion Control For this task we ll use Git an open source tool widely adopted in the industry that is great for modeling parallel work and versioning code files Git enables us to use standard CI CD tools recover previous work with a click of a button insulate the working environment and more If you want to dive into Git for Data Science Martin explains it perfectly in his post But what about the non code artifacts Let s say that the data processing team created a new data set should they just overwrite the old version Or maybe stone age version it by creating a new folder and giving it a meaningful name e g new data v fixed best csv Well both options are inadequate This is why we d want to also version our large files Git was designed for managing software development projects and for versioning text code files Therefore Git doesn t handle large files Git released Git LFS Large File System to overcome large file versioning which is better than Git but fails when scaling Also both Git and Git LFS are not optimized for data science workflow To overcome this challenge many powerful tools emerged in recent years such as DVC Delta Lake LakeFS and more At DagsHub we re integrated with DVC which I love using First and foremost it s open source It provides pipeline capabilities and supports many cloud providers for remote storage Also DVC acts as an extension to Git which allows you to keep using the standard Git flow in your work If you don t want to use both tools I recommend using FDS an open source tool that makes version control for machine learning fast amp easy It combines Git and DVC under one roof and takes care of code data and model versioning Bias alert DagsHub developed FDS Capabilities UnlockedReproducibility ーHaving the state of all the project components under one version enables us to reproduce results with a click of a button Avoid ad hoc versioning done by filenames The code outputs to the same file and versions it Avoid overwriting work because of small mistakes and have the ability to recover files easily Sharing amp synchronizing versions make collaborating with others easier Experiment TrackingMoving our code to functions makes it easier to control the input they get This helps us manage the configuration of every run and track the hyperparameters of each experiment easily We ll move the configurations to an external file and each module will import its relevant variables But as you can imagine tracking each experiment with Git can become a hassle We d like to automate the logging process of each run The same as for large file versioning many tools emerged in recent years for experiment logging such as W amp B MLflow TensorBoard and the list goes on In this case I believe that it doesn t matter with which hammer you choose to hit the nail as long as you punch it through Note that when using external tools you only log the experiment information without the ability to reproduce it Therefore I recommend Mix n Match between the tool and the use case Prototyping a new hypothesis will probably not provide meaningful results therefore we ll use the external tool to log the experiment However after reaching a meaningful result we ll version all of the project components and encapsulate them under a Git commit making it reproducible with a click of a button Capabilities UnlockedResearch history ーAutomating the experiment tracking process will assure logging of all the research history enable us to understand the high level results and avoid repeating our work twice Sharing ーShare the experiment results with the team while having the ability to compare apples to apples Reproducibility ーreproduce the experiments to some extent Avoid ad hoc recording of experiments which will vary between the team members Export Logic to ScriptsJust before we deploy to production we d like to move all of the logic steps stored in the notebook to python scripts However we wouldn t want to lose the notebook capabilities therefore we ll import them back to the notebook Capabilities UnlockedUse only scripts in production which is the industry standard and much easier to handle Maintain one code base and not in both notebook and scripts Keep using notebooks and utilizing their advantages Unit TestsThe same as in software development we d want to make sure that our code model does whatever it was designed to do It goes with data processing modeling and all pipeline stages Writing tests during coding actually makes you work faster overall Since you get a tight fast feedback loop that catches bugs early it s much easier to debug problems since you can debug each unit in isolation instead of debugging a whole running system Capabilities UnlockedFewer bugs in production More sleeping hours SummaryJupyter notebook is an excellent tool for prototyping and exploratory analysis that after all gives us superhuman abilities However when working in a production oriented environment its limitations are revealed When a new project comes in the funnel I highly recommend thinking about the production considerations what tools should be used in the project and how you can make your work more efficient You might measure your work by the model s performance but you are measured by the iteration time to and quality in production If you re facing challenges that I didn t mention in this post or thinking of better ways to overcome them ーI d love to hear about it Feel free to reach out on LinkedIn and Twitter	2022-02-08 11:14:48
海外TECH	CodeProject Latest Articles	VB.NET MVVM Toolkit Demo	https://www.codeproject.com/Tips/5324180/VB-NET-MVVM-Toolkit-Demo	toolkit	2022-02-08 11:56:00
ニュース	BBC News - Home	Calls for windfall tax grow as BP's profits surge	https://www.bbc.co.uk/news/business-60299886?at_medium=RSS&at_campaign=KARANGA	prices	2022-02-08 11:02:54
ニュース	BBC News - Home	Bamber Gascoigne: Original University Challenge presenter dies at 87	https://www.bbc.co.uk/news/entertainment-arts-60301687?at_medium=RSS&at_campaign=KARANGA	bamber	2022-02-08 11:22:28
ニュース	BBC News - Home	EuroMillions: UK ticketholder claims £109.9m jackpot	https://www.bbc.co.uk/news/uk-60302895?at_medium=RSS&at_campaign=KARANGA	claims	2022-02-08 11:03:13
ニュース	BBC News - Home	Canada truckers protest: Trudeau demands an end to trucker protest	https://www.bbc.co.uk/news/world-us-canada-60300163?at_medium=RSS&at_campaign=KARANGA	ottawa	2022-02-08 11:12:52
ニュース	BBC News - Home	Women's Ashes: What we learned from Australia's 12-4 series win over England	https://www.bbc.co.uk/sport/cricket/60299178?at_medium=RSS&at_campaign=KARANGA	Women x s Ashes What we learned from Australia x s series win over EnglandWhy Australia are stacked in all departments and look the side to beat in the forthcoming over World Cup after an Ashes demolition of England	2022-02-08 11:39:49
ニュース	BBC News - Home	How do I book my Covid jab and does it work against Omicron?	https://www.bbc.co.uk/news/health-55045639?at_medium=RSS&at_campaign=KARANGA	england	2022-02-08 11:32:05
北海道	北海道新聞	ダンプ足りない、進まぬ除排雪　豪雪の札幌圏、機械や人手「取り合い」激化	https://www.hokkaido-np.co.jp/article/643369/	取り合い	2022-02-08 20:09:30
北海道	北海道新聞	カラス死骸からＡ型鳥インフル　標津	https://www.hokkaido-np.co.jp/article/643459/	根室管内	2022-02-08 20:15:00
北海道	北海道新聞	寿都核ごみ議事録開示訴訟　全員協議会の「取り決め」巡り対立　判決は３月２９日　函館地裁	https://www.hokkaido-np.co.jp/article/643458/	函館地裁	2022-02-08 20:14:00
北海道	北海道新聞	枋木司４４位、尾崎光輔が８２位　バイアスロン・８日	https://www.hokkaido-np.co.jp/article/643453/	自衛隊	2022-02-08 20:09:00
北海道	北海道新聞	新型コロナ、空港で陽性ゼロ　「バブル」内６人が陽性	https://www.hokkaido-np.co.jp/article/643452/	北京冬季五輪	2022-02-08 20:09:00
北海道	北海道新聞	米、日本の鉄鋼に無関税輸入枠　年１２５万トン、メーカーは歓迎	https://www.hokkaido-np.co.jp/article/643451/	関税	2022-02-08 20:06:00
北海道	北海道新聞	大雪で家庭ごみ収集への影響続く　札幌市内　９日は白石、厚別区で自粛呼び掛け	https://www.hokkaido-np.co.jp/article/643385/	交通障害	2022-02-08 20:02:27
ビジネス	東洋経済オンライン	不眠症に悩む2人が粗大ゴミに目を輝かせたワケ 漫画｢君は放課後インソムニア｣第1集･第5話 | 君は放課後インソムニア | 東洋経済オンライン	https://toyokeizai.net/articles/-/504494?utm_source=rss&utm_medium=http&utm_campaign=link_back	富士山さんは思春期	2022-02-08 20:30:00
マーケティング	AdverTimes	ヨネックス、新社長にヨネヤマ取締役マーケ本部長（22年4月1日付）	https://www.advertimes.com/20220208/article376537/	社長	2022-02-08 11:18:04
海外TECH	reddit	A vegetarian/vegan rant	https://www.reddit.com/r/japanlife/comments/snhoj7/a_vegetarianvegan_rant/	A vegetarian vegan rantI kinda felt the need to get this off my chest But living as a vegan in Japan sucks hard Sure That s probably well known But I want to give some insight into how hard it sucks Because it feels like Japan is actively trying to make life for vegans and vegetarians harder But let me start When you live in Japan and eat meat you probably won t notice all of this stuff at all But when you start looking for it it feels a little bit sick how extremely Japan loves meat I am not talking about stuff here like there being meat advertisements on every corner It s more about how in Japan EVERYTHING has meat pumped into it Even stuff where no sane person would ever expect meat And at some points it feels like they just want to tell vegans quot Fuck you quot An example As a vegetarian your last resort when eating with friends is the good old Pizza Margherita A plain Pizza with nothing but the basics Tomato sauce and cheese It s also offered at a lot of restaurants which aren t very accomodating for vegetarians But if you think you can do that in Japan think again Because Pizza Margherita in this country isn t always made without meat When I recently went into the supermarket I saw a new Pizza Margherita with cheese in the edges Thought I found something I can buy But nooope When checking the ingredients I noticed Japan decided for no fucking reason whatsoever that they should add fucking chicken powder to that thing And that s not a one off There are crackers here with added meat powder Simple tofu sticks made with fish And even potato fries with the added soul of an animal I shit you not No matter whatever entirely plant based product I buy here I check the ingredients twice at least And when I don t see meat on there I am still a little bit unsure In Japan quot meal quot is equivalent with meat Something without meat probably violates some food regulation here which states that plants alone aren t considered safe for human consumption Which could explain why a melobpan with meat filling exists But it goes further Soy products I am talking specifically created soy patties for burgers and so on That shit was only invented because of vegetarians and vegans Guess what Japan does It literally makes soy patties and then adds chicken meat to them What the actual fuck I saw products in the supermarket labeled quot Soy life quot with a big green symbol they had fucking chicken in them And contrary to popular belief Chicken isn t vegan You are going to lose your vegan powers when eating that You ever went to MosBurger Seems like they are accomodating to vegans They have Soy Burgers But actually quot No Fuck you filthy vegan Soy burgers come with meat sauce quot Who even thought of this Who is the target audience of these burgers I seriously don t get it Are they aiming for people trying to eat less calories But they wouldn t go to a fast food place in the first place People eating meat anyways on the other hand would simply take a meat patty Why on earth does something like a Soy Burger with Meat sauce exist The only actual vegan burger MosBurger has is the Green Burger But I am not finished Some stores started ofering vegan products Kind of Burger King with the plant based Burger Lotteria has an option too Taiko Udon has plant based ramen But now in our modern contact less times stuff like Wolt and Uber exists And all these stores despite offering said food in the store itself only offer it on Uber and Wolt during a full moon or not at all On one day I can order my plant based Burger from Burger King via Wolt On the next I can t I am not talking quot Sold out quot here It simply entirely vanishes from the menu Then it comes back then it s gone again Seriously Why And other stores follow suit Konbinis Oh god how I started to hate them It feels like they are teasing me Seven and Lawson always have or different kind of meat burgers but vegan burgers They were available for a single month in and never came back This kinda feels like that they know a vegan market exists but think vegans only need to eat once every years The rest of the time Photosynthesis I guess Even with their other products which aren t specifically targeted at vegans it sucks so hard Because try to find a single meal there without meat Sometimes Seven has some basil pasta as the sole option without meat But I guess they noticed they forgot the meat and have since added tuna to it Every single time you find something without meat it will be gone and never heard of again a week later And it can t be that it doesn t sell When Lawson had their Soy Burgers I had to run to or different stores to find one which still had some in stock Ok I guess it might be different in Tokyo or Osaka But outside of these mega cities you really feel like second class when you try to skip the meat So seriously Japan Stop trying to trick vegans into eating meat every step along the way It s really not that I want everybody to become vegan here it s simply about that I d like to at least have some products without meat I can buy and would also prefer it to not have meat pumped into every single product that originally was vegan As it stands I d even like to have the ingredients list for an apple here because I am sure somebody already thought about giving those a meat layer submitted by u camelization to r japanlife link comments	2022-02-08 11:08:20