MAP-SCTNet is an efficient segmentation network for colorectal cancer histopathological images that improves the SCTNet architecture with three innovations. The first MS-ASPP module uses five parallel dilated convolutions with dual attention and boundary refinement to extract multiscale features while preserving edges. The second AFE-TAM module in the frequency domain combines low-, mid-, and high-frequency components with Gabor filtering and local binary patterns for robust texture representation. The third PDTKD framework uses transformer-based teacher networks and CNNs in three-stage training to improve generalization from limited data. Experiments on the EBHI-SEG set with colonoscopy biopsy samples showed an average cube coefficient of 83.76% and an IoU of 74.5%, which outperformed methods such as TransUNet or SegNet. The model has only 21.8 million parameters and 30.7 G FLOPs, which is 79.3% fewer parameters and 71.1% fewer calculations compared to TransUNet, suitable for real-time diagnostics.