KgSum is a Python application for extracting, preparing, and classifying Knowledge Graphs (KGs). It combines Large Language Models (such as Mistral Instructor 7B with QLoRA) and traditional machine learning for effective graph classification and profiling.
\nFollow these steps to set up KgSum locally.
\nClone the repository:
\ngit clone https://github.com/mariocosenza/kgsum.git\ncd kgsum\n
Create and activate conda environment:
\nconda env create -f environment.yml\nconda activate kgsum\n
For GPU/Transformer Models (Mistral):
\nenvironment.ymlInstall dependencies:
\nnpm install\n
Run the frontend:
\nnpm run dev\n
For GraphDB embedding visualization:
\nsecurity-config.xml with the one in /docker/graphdbSet the following environment variables in your shell:
\nexport GEMINI_API_KEY=your_gemini_api_key_here\nexport LOCAL_ENDPOINT_LOV=http://your-local-endpoint\nexport LOCAL_ENDPOINT=http://your-local-endpoint\nexport SECRET_KEY=your_secret_key_here\nexport UPLOAD_FOLDER=/path/to/uploads\nexport UPLOAD=true\nexport NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=your_clerk_publishable_key\nexport CLASSIFICATION_API_URL=http://localhost:5000\nexport GITHUB_TOKEN=your_github_token_here\n
Configure the backend by editing config.json:
{\n \"labeling\" : {\n \"use_gemini\": false,\n \"search_zenodo\": true,\n \"search_github\": true,\n \"search_lod_cloud\": true,\n \"stop_before_merging\": false\n },\n \"extraction\": {\n \"start_offset\": 0,\n \"step_numbers\": 10,\n \"step_range\": 16,\n \"extract_sparql\": true,\n \"query_lov\": false\n },\n \"processing\" : {\n \"use_ner\": false,\n \"use_filter\": true\n },\n \"training\" : {\n \"classifier\": \"NAIVE_BAYES\",\n \"feature\": [\"CURI\", \"PURI\", \"LAB\", \"CON\", \"TLDS\", \"VOC\", \"LCN\", \"LPN\", \"DSC\", \"SBJ\"],\n \"oversample\": true,\n \"max_token\": 36000,\n \"use_tfidf_autoencoder\": true\n },\n \"profile\": {\n \"store_profile_after_training\": false,\n \"base_domain\": \"http://www.isislab.it\"\n },\n \"general_settings\": {\n \"info\": \"Possible classifiers: SVM, NAIVE_BAYES, KNN, J48, MISTRAL, MLP, DEEP, BATCHNORM, Phase: LABELING, EXTRACTION, PROCESSING, TRAINING, STORE\",\n \"start_phase\": \"labeling\",\n \"stop_phase\": \"training\",\n \"allow_upload\": true\n }\n}\nAvailable Classifiers: SVM, NAIVE_BAYES, KNN, J48, MISTRAL, MLP, DEEP, BATCHNORM
Available Features: CURI, PURI
Processing Phases: LABELING, EXTRACTION, PROCESSING, TRAINING, STORE
Run the complete training process from extraction to model training:
\npython train.py\n
For more fine-tuned control, run individual scripts in /src:
# Run scripts in /src directory for specific phases\n
After completing training, start the WSGI Flask server on port 5000:
\npython app.py\n
Send POST requests to:
\n/api/v1/profile/sparql/api/v1/profile/fileRefer to the Swagger documentation for detailed request and response formats.
\n\nFor a simpler deployment using the pre-trained Naive Bayes model:
\nNavigate to the docker directory:
\ncd /docker\n
Fill the .env file with your configuration
Run with Docker Compose:
\ndocker-compose up\n
Three individual Dockerfiles are provided for custom deployments:
\n| Component | \nSpecification | \n
|---|---|
| CPU | \nAMD Ryzen 5800x | \n
| RAM | \n32 GB DDR4 3600MHz | \n
| GPU | \nNVIDIA RTX 3070 | \n
| Component | \nSpecification | \n
|---|---|
| RAM | \n64+ GB (larger size suggested) | \n
| GPU | \nHigh-performance GPU for better LLM performance | \n
See the open issues for a full list of proposed features (and known issues).
\n\nContributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
\nIf you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag \"enhancement\". Don't forget to give the project a star! Thanks again!
\ngit checkout -b feature/AmazingFeature)git commit -m 'Add some AmazingFeature')git push origin feature/AmazingFeature)