[Could not find the bibliography file(s)

Sirius [?] is an open-source end-to-end standalone intelligent personal assistant (IPA) service. Sirius receives queries in the form of speech or images and returns results in the form of natural language. Sirius implements the core functionalities of an IPA including speech recognition, image matching, natural language processing and a question-and-answer system.

Sirius Intelligent Personal Assistant


Visit our Downloads page to get the latest version of Sirius [?].


Sirius [?] has been tested on Ubuntu 12.04 and 14.04. The following instructions are based from the sirius-application/ directory.


Sirius (and Sirius-suite) [?] has several dependencies which can be resolved with the included get-dependencies.sh. The full list of dependencies is summarized below:

Setting up Sirius

$ tar xzf sirius-1.0.1.tar.gz
$ cd sirius/sirius-application
$ tar xzf question-answer.tar.gz
$ sudo ./get-dependencies.sh
$ sudo ./get-opencv.sh
$ ./get-kaldi.sh
$ ./compile-sirius-servers.sh

After all the correct dependencies have been resolved compile-sirius-servers.sh compiles all the Sirius servers.

The subsequent scripts mentioned are included in the run-scripts/ directory unless otherwise noted.

Automatic Speech Recognition (ASR)

Sirius supports three backends: Kaldi (DNN/HMM based), Pocketsphinx, and Sphinx4 (the latter are GMM/HMM based) to perform Automatic Speech Recognition.

To open an ASR server:

$ ./start-asr-server.sh
$ ./start-asr-server.sh pocketsphinx

or specify an ASR, hostname and port:

$ ./start-asr-server.sh pocketsphinx localhost 8080

In a separate terminal, to test ASR:

$ ./sirius-asr-test.sh ../inputs/questions/what.is.the.speed.of.light.wav

Image Matching (IMM)

Image Matching uses SURF to match query images to a stored database.

In image-matching/ first build and store a database of descriptors in protobuf format where the arguments are the name of the database and the directory containing the images:

$ ./make-db.py landmarks matching/landmarks/db/

To change the database used by the IMM service, change the name in  start-imm-server.py.

In run-scripts/, open the IMM server:

$ ./start-imm-server.sh

In a separate terminal, test IMM using:

$ ./sirius-imm-test.sh ../image-matching/matching/landmarks/query/query.jpg

Question-Answering System (QA)

The Question-Answering system uses OpenEphyra and a Wikipedia database stored in Lemur’s Indri format.

Extract the Wikipedia database (after untaring and building question-answer):

$ wget http://web.eecs.umich.edu/~jahausw/download/wiki_indri_index.tar.gz
$ tar xzvf wiki_indri_index.tar.gz -C question-answer/

In run-scripts/, open the QA server:

$ ./start-qa-server.sh

In a separate terminal, test QA using:

$ ./sirius-qa-test.sh "what is the speed of light"

Combining Services

It is very easy with Sirius to combine ASR and QA to create the full intelligent personal assistant pipeline. After opening multiple servers using ./start-<service>-server.sh, test an ASR-QA query using:

$ ./sirius-asr-qa-test.sh ../inputs/real/what.is.the.capital.of.italy.wav


We would love to have your help in improving Sirius. For questions and comments, post to sirius-users.

Citing Sirius

If you use Sirius in your research, please cite the official publication [?].