The study presents PoSAM-ULTRA, an improved framework for segmenting MRI images of brain tumors, especially gliomas, that improves accuracy and robustness. The model uses the Polar-Bear Foraging Optimization (PBFO) algorithm for hyperparameter tuning and is based on a modified Segment Anything Model with a ResNet-34 encoder for four-channel input. Features are extracted at multiple scales using DownBlocks, discriminative functions are enhanced by the Convolutional Block Attention Module (CBAM), and Attention Gates ensure efficient connection skipping. The multi-stage decoder integrates functions for robust resampling. The model was tested on data from the Integral Genomic Analysis of Low Grade Diffuse Gliomas (LGG) and compared with UNet, UNet++ and nnUNet. PoSAM-ULTRA achieved a cube score of 91.4%, an IoU of 95.2%, a precision of 93.3%, and a recall of 88%. The results confirm its reliability in complex tasks of segmenting medical images of brain tumors.