alexrvandam/SAM2-PAL: SAM2-PAL: SAM2-Palindrome Self-Training with Cycle Consistency

Multi-object palindrome-based self-training for efficient batch segmentation of morphological structures in taxonomic specimens. This implementation is inspired by the OC-CCL (One-Click Cycle-Consistent Learning) methodology described in "Static Segmentation by Tracking" (Feng et al., 2025). This is an original implementation with extensions for multi-object and multi-template training, integrated with the Descriptron GUI (Van Dam & Štarhová Serbina, 2025) for taxonomic digitization workflows.

🎯 What is This? SAM2-PAL enables you to:

Segment hundreds of specimens from a single annotated template Train on multiple structures simultaneously (e.g., scape, antenna, eye) Use multiple diverse templates for better generalization Fine-tune SAM2 using palindrome-based cycle-consistent self-training Integrate with Descriptron GUI for annotation workflows

Key advantage: Instead of training 3 separate models for 3 structures (scape, antenna, eye), train once and segment all structures simultaneously.

🔬 Methodology The Palindrome Approach The core training uses a 4-frame palindrome sequence with memory reset: {x₀, x₁, x₁†, x₀†}

Phase 1: Forward Prediction x₀ (template with mask) → stored in memory x₁ (unlabeled specimen) → predict mask using template

[MEMORY RESET] ← Clears memory to prevent cheating

Phase 2: Backward Verification x₁† (unlabeled with predicted mask) → stored in memory x₀† (template) → predict mask using unlabeled prediction

Loss: Compare final prediction to original ground truth Backprop: Update SAM2 weights end-to-end Why this works:

Memory reset prevents memorization, forces generalization Cycle consistency ensures forward and backward predictions agree Differentiable predictions allow gradients to flow through entire sequence Self-training leverages unlabeled data efficiently

Multi-Object Extension For multiple structures on the same template: For each unlabeled specimen: For each structure (scape, antenna, eye): Run palindrome sequence Compute structure-specific loss

Total loss = sum of all structure losses Backprop once to update shared SAM2 backbone Advantage: Structures share learned features (edges, textures, morphology), improving quality while reducing training time.

Multi-Template Support Instead of one template, use multiple diverse specimens: Training loop: For each unlabeled specimen: Randomly sample a template from [template_1, ..., template_N] Run palindrome training with sampled template Advantage: Exposes model to variation in orientation, lighting, morphology → better generalization.