soyuj/deeper-impact

Paper: DeeperImpact: Optimizing Sparse Learned Index Structures

This repository contains the DeeperImpact model trained on the MS-MARCO passage dataset expanded using a fine-tuned Llama 2 model with hard negatives, distillation, and pre-trained CoCondenser model initialization.

The code to train and run inferences using DeeperImpact can be found in the DeeperImpact Repo.

Please refer to the following notebook to understand how to use the model: inference_deeper_impact.ipynb

For running inference on a larger collection of documents, use the following command:

python -m src.deep_impact.index \
  --collection_path <expanded_collection.tsv> \
  --output_file_path <path> \
  --model_checkpoint_path soyuj/deeper-impact \
  --num_processes <n> \
  --process_batch_size <process_batch_size> \
  --model_batch_size <model_batch_size>

It distributes the inference across multiple GPUs in the machine. To manually set the GPUs, use CUDA_VISIBLE_DEVICES environment variable.