Add Sentence Transformers support

1_Pooling/config.json

@@ -0,0 +1,9 @@
+{
+  "word_embedding_dimension": 2048,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false
+}

README.md

@@ -3,6 +3,8 @@ license: apache-2.0
 datasets:
 - bigcode/the-stack-dedup
 library_name: transformers
+tags:
+- sentence-transformers
 language:
 - code
 ---
@@ -24,6 +26,8 @@ This checkpoint is first trained on code data via masked language modeling (MLM)
 ### How to use
 This checkpoint consists of an encoder (1.3B model), which can be used to extract code embeddings of 2048 dimension. It can be easily loaded using the AutoModel functionality and employs the Starcoder tokenizer (https://arxiv.org/pdf/2305.06161.pdf).
 
+### Transformers
+
 ```
 from transformers import AutoModel, AutoTokenizer
 
@@ -39,6 +43,21 @@ print(f'Dimension of the embedding: {embedding[0].size()}')
 # Dimension of the embedding: torch.Size([13, 2048])
 ```
 
+### Sentence Transformers
+
+```
+from sentence_transformers import SentenceTransformer
+
+checkpoint = "codesage/codesage-large"
+device = "cuda"  # for GPU usage or "cpu" for CPU usage
+
+model = SentenceTransformer(checkpoint, device=device, trust_remote_code=True)
+
+embedding = model.encode("def print_hello_world():\tprint('Hello World!')")
+print(f'Dimension of the embedding: {embedding.size}')
+# Dimension of the embedding: 2048
+```
+
 ### BibTeX entry and citation info
 ```
 @inproceedings{

config_sentence_transformers.json

@@ -0,0 +1,7 @@
+{
+  "__version__": {
+    "sentence_transformers": "2.4.0.dev0",
+    "transformers": "4.37.0",
+    "pytorch": "2.1.0+cu121"
+  }
+}

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]