c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,430 Commits 171 Branches 49 Tags
40bb99e1be65490bc5018ca8cae69f641f53296b
Commit Graph
68 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Morgan Funtowicz 7ac47bfe69 Updated notebook dependencies for Colab. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-03-05 16:07:51 +01:00
Morgan Funtowicz be02176a4b Fixing sentiment pipeline in 03-pipelines notebook. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-03-05 16:07:51 +01:00
Morgan Funtowicz 012cbdb0f5 Updating colab links in notebooks README. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-03-05 15:34:15 +01:00
Morgan Funtowicz 30624f7056 Fix Colab links + install dependencies first. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-03-05 11:40:15 +01:00
Morgan Funtowicz 1bca97ec7f Update notebook link and fix few working issues. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-03-04 21:19:33 +01:00
Morgan Funtowicz 0e7d609a8b Trying to lower the number of parallel runs to avoid flaky TF QA pipelines tests 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-02-24 16:12:06 +01:00
Morgan Funtowicz 2493998ae0 Reenable tf_qa_pipelines for debugging purpose. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-02-24 15:22:45 +01:00
Morgan Funtowicz 6aa7973aec Fix circleci cuInit error on Tensorflow >= 2.1.0. 
Tensorflow 2.1.0 introduce a new dependency model where pip install tensorflow would install tf with GPU support.
Before it would just install with CPU support, thus CircleCI is looking for NVidia driver version at initialization of the
tensorflow related tests but fails as their is no NVidia Driver running.

Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-02-10 13:24:37 +01:00
Morgan Funtowicz 908cd5ea27 Make forward asynchrone to avoid long computation timing out. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-01-20 10:56:23 -05:00
Morgan Funtowicz 6e6c8c52ed Fix bad handling of env variable USE_TF / USE_TORCH leading to invalid framework being used. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-01-20 10:56:23 -05:00
Morgan Funtowicz 0d6c17fc1b black formatting 2020-01-13 11:18:27 +01:00
Morgan Funtowicz 9261c7f771 Remove f-string device creation on PyTorch GPU pipelines. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-01-07 11:46:44 +01:00
Morgan Funtowicz 91d33c798b Fix issue on pipelines where pytorch's tensors are not copied on the user-specified GPU device. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-01-07 11:12:31 +01:00
Morgan Funtowicz 80faf22b4a Updating documentation for converting tensorflow model to reflect the new cli convert format. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-01-04 13:41:18 +01:00
Morgan Funtowicz b98ff88544 Added pipelines quick tour in README 2019-12-20 15:52:50 +01:00
Morgan Funtowicz 7f74084528 Fix leading axis added when saving through the command run 2019-12-20 14:47:04 +01:00
Morgan Funtowicz 61d9ee45e3 All tests are green. 2019-12-20 11:47:56 +01:00
Morgan Funtowicz e516a34a15 Use BasicTokenizer to split over whitespaces. 2019-12-20 09:38:08 +01:00
Morgan Funtowicz 9d0d1cd339 Filter out entity for NER task. 2019-12-20 09:30:37 +01:00
Morgan Funtowicz a305067f2d Removed __main__ 2019-12-19 19:41:48 +01:00
Morgan Funtowicz 3492a6ec17 Addressing Thom's comments. 2019-12-19 19:06:44 +01:00
Morgan Funtowicz 81a911cce5 Doc, doc, ... doc. 2019-12-19 15:12:06 +01:00
Morgan Funtowicz faef6f6191 Fix logic order for USE_TF/USE_TORCH 2019-12-19 12:28:17 +01:00
Morgan Funtowicz 5664327c24 Hide train command for now. 2019-12-19 12:27:54 +01:00
Morgan Funtowicz 3b29322d4c Expose all the pipeline argument on serve command. 2019-12-19 12:24:17 +01:00
Morgan Funtowicz fc624716aa Renaming framework env variables flags from NO_ to USE_ 2019-12-19 11:49:06 +01:00
Morgan Funtowicz f516cf3956 Allow pipeline to write output in binary format 2019-12-19 11:42:33 +01:00
Morgan Funtowicz d72fa2a0f6 Fix inputs_for_model call in QuestionAnsweringPipeline accessing __dict__ on list. 2019-12-19 10:54:10 +01:00
Morgan Funtowicz bcc99fd92e Fix wrong automatic config allocation through AutoConfig 2019-12-19 10:32:21 +01:00
Morgan Funtowicz ec5d6c6a70 Adressing issue with NER task omitting first and last word. 2019-12-19 00:12:10 +01:00
Morgan Funtowicz d0724d0794 Add PipedPipelineDataFormat 2019-12-18 23:27:26 +01:00
Morgan Funtowicz 7711403bbd Expose config through the cli arguments 2019-12-18 22:59:51 +01:00
Morgan Funtowicz 8bb166db5d Expose more information in the output of TextClassificationPipeline 2019-12-18 22:53:19 +01:00
Morgan Funtowicz 04b602f96f Put module import on top of the module. 2019-12-18 18:28:39 +01:00
Morgan Funtowicz 0c88c856d5 Unnest QuestionAnsweringArgumentHandler 2019-12-18 18:18:16 +01:00
Morgan Funtowicz e347725d8c More fine-grained control over pipeline creation with config argument. 2019-12-18 10:41:24 +01:00
Morgan Funtowicz 2fde5a2489 Initial bunch of documentation. 2019-12-17 12:16:07 +01:00
Morgan Funtowicz d7c62661a3 Provide serving dependencies for tensorflow and pytorch (serving-tf, serving-torch) 2019-12-17 11:23:39 +01:00
Morgan Funtowicz a096e2a88b WIP serving through HTTP internally using pipelines. 2019-12-16 16:38:02 +01:00
Morgan Funtowicz 43a4e1bbe4 Adressing issue in varargs handling for question answering. 2019-12-16 16:00:41 +01:00
Morgan Funtowicz 46ccbb42fc Make CLI run command use integer mapping for device argument. 2019-12-16 15:49:41 +01:00
Morgan Funtowicz bbc707cf39 Fix non-keyworded varargs handling in DefaultArgumentHandler for pipeline. 2019-12-16 15:49:09 +01:00
Morgan Funtowicz 9c391277cc Allow tensors placement on specific device through CLI and pipeline. 2019-12-16 15:19:13 +01:00
Morgan Funtowicz 955d7ecb57 Refactored Pipeline with dedicated argument handler. 2019-12-16 14:34:54 +01:00
Morgan Funtowicz 8e3b1c860f Added FeatureExtraction pipeline. 2019-12-15 01:37:52 +01:00
Morgan Funtowicz f1971bf303 Binding pipelines to the cli. 2019-12-15 01:37:16 +01:00
Morgan Funtowicz 0b51532ce9 Reintroducing the batch_encode_plus method 2019-12-13 16:22:50 +01:00
Morgan Funtowicz 8938b546bf Removed from_config 2019-12-13 14:27:04 +01:00
Morgan Funtowicz 1ca52567a4 Allow model conversion in the pipeline allocator. 2019-12-13 14:13:14 +01:00
Morgan Funtowicz 28e64ad5a4 Raise an exception if the pipeline allocator can't determine the tokenizer from the model. 2019-12-13 14:12:54 +01:00
Morgan Funtowicz be5bf7b81b Added NER pipeline. 2019-12-13 14:12:17 +01:00
Morgan Funtowicz 80eacb8f16 Adding labels mapping for classification models in their respective config. 2019-12-13 14:10:22 +01:00
Morgan Funtowicz f69dbecc38 Expose classification labels mapping (and reverse) in model config. 2019-12-12 10:25:36 +01:00
Morgan Funtowicz c28273793e Add missing DistilBert and Roberta to AutoModelForTokenClassification 2019-12-11 15:31:45 +01:00
Morgan Funtowicz b040bff6df Added supported model to AutoModelTokenClassification 2019-12-11 14:13:58 +01:00
Morgan Funtowicz 9a24e0cf76 Refactored qa pipeline argument handling + unittests 2019-12-11 00:33:25 +01:00
Morgan Funtowicz 63e36007ee Make sure padding, cls and another non-context tokens cannot appear in the answer. 2019-12-10 16:47:35 +01:00
Morgan Funtowicz 40a39ab650 Reuse recent SQuAD refactored data structure inside QA pipelines. 2019-12-10 15:59:38 +01:00
Morgan Funtowicz aae74065df Added QuestionAnsweringPipeline unit tests. 2019-12-10 13:37:20 +01:00
Morgan Funtowicz a7d3794a29 Remove token_type_ids for compatibility with DistilBert 2019-12-10 13:37:20 +01:00
Morgan Funtowicz fe0f552e00 Use attention_mask everywhere. 2019-12-10 13:37:20 +01:00
Morgan Funtowicz 348e19aa21 Expose attention_masks and input_lengths arguments to batch_encode_plus 2019-12-10 13:37:18 +01:00
Morgan Funtowicz c2407fdd88 Enable the Tensorflow backend. 2019-12-10 13:37:14 +01:00
Morgan Funtowicz f116cf599c Allow hidding frameworks through environment variables (NO_TF, NO_TORCH). 2019-12-10 13:37:07 +01:00
Morgan Funtowicz 6e61e06051 batch_encode_plus generates the encoder_attention_mask to avoid attending over padded values. 2019-12-10 13:37:07 +01:00
Morgan Funtowicz 02110485b0 Added batching, topk, chars index and scores. 2019-12-10 13:36:55 +01:00
Morgan Funtowicz e1d89cb24d Added QuestionAnsweringPipeline with batch support. 2019-12-10 13:36:55 +01:00
Morgan Funtowicz 81babb227e Added download command through the cli. 
It allows to predownload models and tokenizers.
 2019-12-10 12:18:59 +01:00