Elevating Deep Learning Projects: Community Insights on Software Project Quality and Reproducibility

Elevating Deep Learning Projects: Community Insights on Software Project Quality and Reproducibility

A recent GitHub discussion initiated by PrahitViraajReddy, a B.Tech AI & Data Science student, sparked a rich conversation on enhancing a deep learning computer vision project. PrahitViraajReddy's project, "Autism Image Classification Using Deep Learning", leverages PyTorch and DenseNet-169 for binary image classification on facial images, complemented by a Streamlit interface for interactive predictions. The author sought specific technical feedback across three crucial areas: model development, software engineering, and overall project quality.

The community's responses provided a wealth of actionable advice, emphasizing robustness, reproducibility, and ethical considerations—all vital components of strong software project quality in AI development.

1. Fortifying Model Development and Evaluation

The feedback highlighted several critical methodological issues:

• Preventing Data Leakage: A recurring theme was the necessity of a patient-level train/validation/test split, especially when multiple images belong to the same individual. This prevents the model from memorizing identities rather than learning generalizable features. Suggestions included using `GroupKFold` on subject IDs and checking for duplicates or near-duplicates across splits using perceptual hashes or embedding similarity.
• Comprehensive Evaluation Metrics: Beyond accuracy, participants recommended reporting precision, recall, F1-score, and confusion matrices. For imbalanced datasets, ROC-AUC and PR-AUC were advised. Evaluating on a truly held-out test set and comparing against simple baselines were also stressed.
• Rigorous Augmentation and Preprocessing: Ensure data augmentation is applied exclusively to the training set, with validation and test transforms being deterministic. Consistency in preprocessing between training and inference was deemed crucial.
• Shortcut Checks and Interpretability: To verify the model learns relevant features, suggestions included training with shuffled labels (to detect leakage if accuracy remains high), evaluating on masked faces or background-only regions, and using Grad-CAM to visualize which facial regions the network focuses on.
• Model Calibration: Softmax scores are often over-confident. Checking reliability diagrams or expected calibration error, and considering temperature scaling, was recommended before presenting outputs as "probabilities."

2. Enhancing Software Engineering Practices

Improving the project's structure, reproducibility, and integration was a key focus for elevating software project quality:

• Modularity and Consistency: Structure the project into distinct modules for data loading, preprocessing, training, evaluation, and Streamlit inference. A single preprocessing module, imported by both `train.py` and `app.py`, ensures consistency.
• Reproducibility: Centrally store configuration values and random seeds. Pin dependencies in `requirements.txt` (including exact Torch and Torchvision versions). Save model weights, class mappings, image sizes, and normalization values together in the checkpoint file.
• Streamlit Optimization: Load the PyTorch model once using `@st.cache_resource` to prevent bottlenecks during UI reruns. Remember to call `model.eval()` with `torch.no_grad()` for efficient inference.
• Robust Input Handling: Implement validation for uploaded image formats, dimensions, and corrupted files. Consider adding a face-detection step or rejecting low-confidence/out-of-distribution inputs, as the model will confidently classify any image.
• Testing and CI: Add basic tests for the prediction pipeline and integrate a GitHub Actions workflow to run them, ensuring continuous validation of the core functionality.

3. Boosting Overall Project Quality and Ethical Considerations

For a stronger computer vision portfolio piece, the community offered advice spanning documentation, transparency, and ethical framing:

• Comprehensive Documentation: Include a model card and dataset card detailing source, labelling method, class counts, splits, and known issues. Add an error-analysis section with example failures.
• Transparency and Disclaimers: Given the sensitive nature of autism image classification, explicitly state dataset provenance, limitations, and known failure cases in the README. For the public demo, clarify what happens to uploaded images (e.g., "not stored or logged") and keep the medical disclaimer visible next to every prediction. The separate Qwen3-1.7B assistant should also be clearly labeled with possible inaccuracies.
• Reframing the Project: Several contributors suggested framing the project as a study of dataset bias and shortcut learning, with the diagnostic checks as the main result. This approach acknowledges the scientific contestation around facial-image autism prediction, making a negative or cautionary result more credible than a high accuracy number. This demonstrates a mature understanding of the domain and contributes significantly to the overall software project quality.

The discussion underscored that for projects involving sensitive data or medical implications, prioritizing leakage-resistant evaluation, reproducibility, and transparent ethical considerations are paramount before adding more model complexity. These practices not only enhance the technical merit but also the credibility and responsible development of AI systems.