[삽질저장용] 머신러닝 모델 서빙서버 만들기

bert, sagemaker, aws, ec2, nboost,lambda

https://pypi.org/project/nboost/

nboost

https://hanxiao.io/2019/07/29/Generic-Neural-Elastic-Search-From-bert-as-service-and-Go-Way-Beyond/#gnes-preliminaries-breakdown-of-neural-elastic-and-search

Generic Neural Elastic Search: From bert-as-service and Go Way Beyond · Han Xiao Tech Blog - Deep Learning, NLP, AI

https://github.com/hanxiao/bert-as-service#using-bert-as-service-to-serve-http-requests-in-json

hanxiao/bert-as-service

http://www.lib4dev.in/info/koursaros-ai/nboost/LICENSE#setting-up-an-elasticsearch-server

Train and Deploy the Mighty BERT based NLP models using FastBert and Amazon SageMaker

https://medium.com/@kaushaltrivedi/train-and-deploy-mighty-transformer-nlp-models-using-fastbert-and-aws-sagemaker-cc4303c51cf3

Train and deploy the Mighty BERT based NLP models using FastBert and Amazon SageMaker

언급된것처럼 gpu 서버 너무 비쌈...

https://beomi.github.io/2018/03/18/Create_GPU_spot_EC2_for_ML/

GPU EC2 스팟 인스턴스에 Cuda/cuDNN와 Tensorflow/PyTorch/Jupyter Notebook 세팅하기 - Beomi's Tech blog

https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/docker.md

tensorflow/serving

간단하게 tf 서빙서버를 만드는 것도 방법이나 bert 를 쓸거면 bert를 쓰는게 나을 거라고 판단. 

http://blog.naver.com/PostView.nhn?blogId=wideeyed&logNo=221566170795&categoryNo=56&parentCategoryNo=0&viewDate=¤tPage=1&postListTopCurrentPage=1&from=search

[Tensorflow] Serving이용하여 REST API 만들기

https://tensorflowkorea.gitbooks.io/tensorflow-kr/content/g3doc/tutorials/tfserve/

텐서플로우 서빙 · 텐서플로우 문서 한글 번역본

https://www.tensorflow.org/tfx/serving/docker

TensorFlow Serving with Docker | TFX

https://www.tensorflow.org/tfx/serving/serving_advanced

Building Standard TensorFlow ModelServer | TFX

https://www.tensorflow.org/guide/saved_model?hl=ko

SavedModel 포맷 사용하기 | TensorFlow Core

https://www.tensorflow.org/tutorials/keras/save_and_load?hl=ko

모델 저장과 복원 | TensorFlow Core

https://zzsza.github.io/data/2018/07/12/tensorflow-serving-tutorial/

Tensorflow Serving Tutorial

https://gusrb.tistory.com/37

[Tensorflow-Serving] 텐서플로우 서빙 설치 하기

https://www.tensorflow.org/tfx/serving/api_rest

RESTful API | TFX | TensorFlow

https://cloud.google.com/solutions/machine-learning/building-real-time-embeddings-similarity-matching-system?hl=ko

실시간 임베딩 유사성 일치 시스템 빌드 | 솔루션 | Google Cloud

보고 깜짝 놀람, 자사 gcp 서비스들을 이용해서 시스템 구축하는걸 다 문서화해놓음; 

gcp 를 썼다면 이런 문서들을 기반으로 text-embedding 시스템을 구축했을 것. 

https://github.com/GoogleCloudPlatform/realtime-embeddings-matching/tree/master/text-semantic-search

GoogleCloudPlatform/realtime-embeddings-matching

aws arn

이걸로 lambda 세팅할 때 도움 많이 받음. ㅠㅠ 물론 이걸 사용하는 일은 없었다고 한다. 

https://github.com/antonpaquin/Tensorflow-Lambda-Layer/blob/master/arn_tables/tensorflow_keras.md

antonpaquin/Tensorflow-Lambda-Layer

https://towardsdatascience.com/word-embedding-using-bert-in-python-dd5a86c00342

Word Embedding Using BERT In Python

https://bert-as-service.readthedocs.io/en/latest/source/server.html

Using BertServer — bert-as-service 1.6.1 documentation

https://github.com/DemisEom/bertsearch

DemisEom/bertsearch

이걸로 쉽게 돌려볼 수 있었을까?

https://github.com/Hironsan/bertsearch

Hironsan/bertsearch

https://bert-as-service.readthedocs.io/en/latest/source/server.html

Using BertServer — bert-as-service 1.6.1 documentation

https://www.elastic.co/kr/blog/text-similarity-search-with-vectors-in-elasticsearch

벡터 필드를 사용한 Elasticsearch의 텍스트 유사도 검색

https://github.com/mikepm35/TfLambdaDemo

mikepm35/TfLambdaDemo

https://medium.com/@mike.p.moritz/running-tensorflow-on-aws-lambda-using-serverless-5acf20e00033

Running TensorFlow on AWS Lambda using Serverless

https://www.youtube.com/watch?v=neHlZCMUN8U

https://www.slideshare.net/awskorea/aws-lambda-tensorflow-keras-inferences

AWS Lambda를 통한 Tensorflow 및 Keras 기반 추론 모델 서비스하기 :: 이준범 :: AWS Summit…

https://mc.ai/serverless-vision-inference-using-aws-lambda-and-tflite/

Serverless Vision inference using AWS Lambda and TFLite

tflite로도 뭔갈 해보려고 했으나 잘 되지 않음. 결국 런타임을 lambda 에서 돌리는데는 무리가 있었음. 

https://m.blog.naver.com/PostView.nhn?blogId=olpaemi&logNo=221639136195&proxyReferer=https:%2F%2Fwww.google.com%2F

엘라스틱서치의 벡터(Vector) 필드와 텐서플로우를 이용한 문서 유사도 검색 (1) > Similarity Search #ela

https://aws.amazon.com/ko/blogs/korea/build-a-serverless-frontend-for-an-amazon-sagemaker-endpoint/

Amazon SageMaker를 위한 서버리스 엔드포인트 만들기 | Amazon Web Services

https://aws.amazon.com/ko/blogs/machine-learning/enhanced-text-classification-and-word-vectors-using-amazon-sagemaker-blazingtext/

Enhanced text classification and word vectors using Amazon SageMaker BlazingText | Amazon Web Services

https://github.com/haandol/mxnet-elastic-inference/blob/master/MXNet%20SSD%20on%20AWS%20Elastic%20Inference.ipynb

haandol/mxnet-elastic-inference

https://medium.com/circuitpeople/serverless-large-file-downloads-to-s3-a11b4ef4788e

Serverless, large file downloads to S3

https://blog.haandol.com/2020/03/07/search-keyword-correction.html

아주 간단한 한국어 검색어 교정기 만들기, Haandol

https://docs.aws.amazon.com/ko_kr/dlami/latest/devguide/tutorial-tfserving.html

TensorFlow Serving - 딥 러닝 AMI

https://gist.github.com/nacyot/8366310

도커(Docker) 치트 시트

삽질한다고 도커 많이 써봄. 

수확이었다. 

https://stackoverflow.com/questions/30604846/docker-error-no-space-left-on-device

Docker error : no space left on device

ec2 t2.micro에다가 이것저것 설치하다보니 마주한 문제. 

https://aws.amazon.com/ko/premiumsupport/knowledge-center/batch-job-failure-disk-space/

AWS Batch에서 "No space left on device(디바이스에 남은 공간 없음)" 오류 해결

https://aws.amazon.com/ko/blogs/machine-learning/deploy-trained-keras-or-tensorflow-models-using-amazon-sagemaker/

Deploy trained Keras or TensorFlow models using Amazon SageMaker | Amazon Web Services

https://tfhub.dev/tensorflow/bert_multi_cased_L-12_H-768_A-12/2

TensorFlow Hub

구글이 최고여

https://urbanbase.github.io/dev/2019/10/24/Machine-Learning-Prototype.html

클라우드 환경에서 머신러닝 서비스 프로토타입 빠르게 만들어보기

https://github.com/harusametime/sagemaker-notebooks/tree/master/tf_bert_classifier/src/bert

harusametime/sagemaker-notebooks

https://github.com/harusametime/sagemaker-notebooks/tree/master/tf_bert_classifier/src/bert#pre-trained-models

harusametime/sagemaker-notebooks

https://github.com/harusametime/sagemaker-notebooks/tree/master/tf_bert_classifier/src/bert#fine-tuning-with-bert

harusametime/sagemaker-notebooks

https://hackernoon.com/exploring-the-aws-lambda-deployment-limits-9a8384b0bec3

How to beat the AWS Lambda deployment limits | Hacker Noon

https://github.com/serverless/serverless

serverless/serverless

https://dreamgonfly.github.io/2018/01/19/pytorch-on-aws-lambda.html

AWS Lambda로 PyTorch 모델 서빙하기

사실 삽질중에 pytorch 서빙서버가 공식으로 aws에서 나와서 좀 고민하긴했음.

https://www.freecodecamp.org/news/what-we-learned-by-serving-machine-learning-models-using-aws-lambda-c70b303404a1/

What We Learned by Serving Machine Learning Models Using AWS Lambda

https://github.com/tgjeon/TF-Eager-Execution-Guide-KR/blob/master/guide.md

tgjeon/TF-Eager-Execution-Guide-KR

https://aws.amazon.com/ko/blogs/machine-learning/how-to-deploy-deep-learning-models-with-aws-lambda-and-tensorflow/

How to Deploy Deep Learning Models with AWS Lambda and Tensorflow | Amazon Web Services

https://www.whatap.io/ko/blog/37/

리눅스 free 명령어로 메모리 상태 확인하기 | 와탭 블로그

https://github.com/mikepm35/TfLambdaDemo

mikepm35/TfLambdaDemo

https://medium.com/smellslikeml/serverless-vision-inference-using-aws-lambda-and-tflite-613c084f64d2

Serverless Vision inference using AWS Lambda and TFLite

https://aws.amazon.com/ko/blogs/machine-learning/deploy-trained-keras-or-tensorflow-models-using-amazon-sagemaker/?nc1=h_ls

Deploy trained Keras or TensorFlow models using Amazon SageMaker | Amazon Web Services

https://github.com/aws/sagemaker-tensorflow-serving-container

aws/sagemaker-tensorflow-serving-container

https://github.com/jina-ai/jina/tree/master/docs/chapters/101

jina-ai/jina

이것도 신기하긴했음. 

https://docs.jina.ai/api/jina.html

Jina AI

https://github.com/jina-ai/examples/tree/master/x-as-service

jina-ai/examples

https://course.fast.ai/#pytorch-and-fastai

Practical Deep Learning for Coders, v3 | fast.ai course v3

http://xplordat.com/2019/10/28/semantics-at-scale-bert-elasticsearch/

Semantics at Scale: BERT + Elasticsearch

https://towardsdatascience.com/how-we-built-an-ai-powered-search-engine-without-being-google-5ad93e5a8591

How we built an AI-powered search engine (without being Google)

https://github.com/ryfeus/lambda-packs

ryfeus/lambda-packs

lambda pack들, 람다 세팅할 때 도움이 많이 됨. 

https://mc.ai/serving-pytorch-nlp-models-on-aws-lambda/

Serving Pytorch NLP models on AWS Lambda

https://www.tensorflow.org/tfx/serving/serving_config#reloading_model_server_configuration

Tensorflow Serving Configuration  |  TFX

https://www.tensorflow.org/tfx/serving/custom_source

Creating a module that discovers new servable paths  |  TFX  |  TensorFlow

https://www.tensorflow.org/tfx/serving/signature_defs#objective

SignatureDefs in SavedModel for TensorFlow Serving  |  TFX

https://github.com/hanxiao/bert-as-service/issues/221

Recommended settings for AWS p2 instance · Issue #221 · hanxiao/bert-as-service

https://towardsdatascience.com/how-to-build-a-semantic-search-engine-in-3-minutes-9e579da92913

How to Build a Semantic Search Engine in 3 minutes

https://towardsdatascience.com/serving-tensorflow-models-serverless-6a39614094ff

Serving TensorFlow Models. Serverless

https://antilibrary.org/2259

텐서플로우 임포팅 경고 메시지 synonym of type is deprecated; in a future version of numpy, it will be understood as

https://github.com/kaushaltrivedi/fast-bert

kaushaltrivedi/fast-bert

fast bert, bert 종류들 많아서 다 살펴볼만