Commit
·
df91210
1
Parent(s):
3b515a1
Update README.md
Browse files
README.md
CHANGED
|
@@ -19,7 +19,7 @@ metrics:
|
|
| 19 |
|
| 20 |
# Multi-purpose Summarizer (Fine-tuned 3B google/flan-t5-xl on several Summarization datasets)
|
| 21 |
|
| 22 |
-
<a href="https://colab.research.google.com/
|
| 23 |
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
|
| 24 |
</a>
|
| 25 |
|
|
@@ -67,18 +67,11 @@ results = summarizer(
|
|
| 67 |
)
|
| 68 |
```
|
| 69 |
|
| 70 |
-
**For Batch Inference:** see [this discussion thread](https://huggingface.co/pszemraj/flan-t5-large-grammar-synthesis/discussions/1) for details, but essentially the dataset consists of several sentences at a time, and so I'd recommend running inference **in the same fashion:** batches of 64-96 tokens ish (or, 2-3 sentences split with regex)
|
| 71 |
-
|
| 72 |
-
- it is also helpful to **first** check whether or not a given sentence needs grammar correction before using the text2text model. You can do this with BERT-type models fine-tuned on CoLA like `textattack/roberta-base-CoLA`
|
| 73 |
-
- I made a notebook demonstrating batch inference [here](https://colab.research.google.com/gist/pszemraj/6e961b08970f98479511bb1e17cdb4f0/batch-grammar-check-correct-demo.ipynb)
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
---
|
| 78 |
|
| 79 |
## Training procedure
|
| 80 |
|
| 81 |
-
- Training was done in BF16, deepspeed stage 2 for 6 epochs with
|
| 82 |
|
| 83 |
## Hardware
|
| 84 |
- GPU count 8 NVIDIA A100-SXM4-40GB
|
|
|
|
| 19 |
|
| 20 |
# Multi-purpose Summarizer (Fine-tuned 3B google/flan-t5-xl on several Summarization datasets)
|
| 21 |
|
| 22 |
+
<a href="https://colab.research.google.com/drive/1EYfnIoG-r5lL2-3oiO_YdYEVKB0pAa9h">
|
| 23 |
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
|
| 24 |
</a>
|
| 25 |
|
|
|
|
| 67 |
)
|
| 68 |
```
|
| 69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
---
|
| 71 |
|
| 72 |
## Training procedure
|
| 73 |
|
| 74 |
+
- Training was done in BF16, deepspeed stage 2 for 6 epochs with ROUGE-2 monitored on the validation set.
|
| 75 |
|
| 76 |
## Hardware
|
| 77 |
- GPU count 8 NVIDIA A100-SXM4-40GB
|