Installation & Setup

There is a dockerized version of the *-DCC available, which can be used during development as well as in production.

Dependecies

  • git >=1.8
  • docker >=18.09
  • docker-compose >=1.23

Development Setup

Start Development Server

  • Clone the git repository: $ git clone https://gitlab.com/danio-code/public/dcc.git
  • Change into the project directory $ cd dcc
  • Build and start the docker image with $ docker-compose up (make sure your docker deamon is running)
  • When you see a the message “Compiled successfully” from webpack, you can inspect the running DCC on http://localhost:3000
  • The admin username is test-user with the password password.

The project follows the general Django project structure.

Backend

  • The methods concerning the backend are located in views.py. The database models are in models.py and urls in urls.py. The html templates are located inside templates/, with base.html, footer.html and menu.html included in every template.

Frontend

  • The images, css and javascript files are in the static/ directory, with their respective subdirectories. The css are compiled from sass files in the same css/ directory. The JavaScript files are transpiled and minified with their source files inside js/. Files written with the React framework are prefixed with an underscore. All these files are transpiled and minified via webpack using the respective webpack.*.config.js configuration files.

Tests

The Django test framework is used for unit tests. The tests are located at dcc/modeldcc/tests.py. To run the unit tests, open an additional terminal window, change into the project directory and run: $ docker exec -it web python dcc/manage.py test modeldcc --nocapture.

Cypress is used for E2E test and the test can be found in cypress/integration. They are called (with the dev-server docker container running in parallel) by: $ npx cypress run.

Production Setup

Deploy

  • Clone the git repository: $ git clone https://gitlab.com/danio-code/public/dcc.git
  • Change into the project directory $ cd dcc
  • Create in the same directoyr a .env file with the following variables (use the values given in the example to run the current docker setup):
SECRET_KEY= # a random string, can be generated by running `$ python generate_secret.py`

SQL_ENGINE= # which sql engine is used, e.g., django.db.backends.postgresql
SQL_DATABASE= # name of your database, e.g. testdata
SQL_USER= # user name to access database with, e.g. test-user
SQL_PASSWORD= # password corresponding to SQL_USER, e.g. duckpin-celsius-phony-glaucous-conspire-lisp-haggard-nucleate
SQL_HOST= # Hostname for database, e.g. postgres
SQL_PORT= # port where database can be reached, e.g. 5432

PYTHONPATH= # set the python path inside the container, e.g./var/www/dcc
DJANGO_SETTINGS_MODULE= # path to settings file in python module syntax, e.g. dcc.settings.production

HOST_EMAIL= # email host used to send emails through, e.g. smtp.gmail.com
HOST_USER= # username for the email
HOST_PASSWD= # password for the email

ADMIN_EMAIL= # email address, where the error logs will be sent to

GROUP= #name of the user group, e.g. test, which is used to restrict the access to certain pages with the has_group template tag
  • Add your domain name or IP address in ALLOWED_HOST in docker-compose-prod.yml (but keep 'web', '0.0.0.0').
  • Build and start the docker image in the background with $ docker-compose -f docker-compose-prod.yml up --build -d (make sure your docker daemon is running)

Change Controlled Vocabularies

If you just want to add terms to a vocabulary, add it in the respective json file in dcc/static/json/ and also to the database. The latter can be done via the admin interface or by directly interacting with the postgres database.

In order to completely exchange a controlled vocabulary, replace the respective json file in dcc/static/json/ and then run $ python load_vocabulary.py (Caution: this will drop the respective tables and any links to other terms will brake. This step is therefor only recommended during setup).

Handling of Annotation and Data Upload

Depending on the file sizes and the internet connection of both the *DCC-server and the annotator, the file upload can last a few minutes up to hours. A success message will appear in the web-form or the batch-upload page, respectively. The platform can handle shorter network disruptions and will continue also if the page has been closed. Successful uploads will appear on the data export page.

An incorrect or incomplete upload can be corrected via the admin interface.