An Efficient Algorithm for Imbalanced Data Streams Management in Opinion Analysis

In many real-world streaming applications, the unequal distribution of class instances presents significant challenges to conventional machine learning algorithms, leading to biased predictions and poor detection of minority-class events. This study proposes an Efficient Algorithm for Imbalanced Data Stream Management that addresses class imbalance while maintaining high processing speed and predictive accuracy in dynamic streaming environments. Existing approaches to opinion analysis in dynamic data streams face notable limitations, particularly in managing imbalanced sentiment distributions and adapting to evolving feature spaces over time. Opinion analysis often deals with imbalanced data streams, where certain sentiments or opinions may be more prevalent than others, which can cause the model to become biased toward the more common results. Addressing imbalanced data streams is crucial to prevent biased models and ensure accurate sentiment analysis. The methodology integrates a transformer-based contextual embedding model (DistilBERT) with an adaptive autoencoder for feature compression and Synthetic Minority Over-Sampling Technique (SMOTE) to address class imbalance in streaming sentiment datasets. Concept drift in dynamic opinion data is monitored using Feature Selection Stability and Drift (FSSD) combined with the ADWIN drift detection algorithm, enabling the system to adapt to evolving sentiment patterns over time. Performance evaluation is conducted using benchmark imbalanced data stream datasets and real-time streaming data, with assessment based on classification accuracy, precision, recall, F1-score, G-mean, Area Under the ROC Curve (AUC), processing time, memory utilization, and scalability. Experimental results demonstrate that the proposed algorithm effectively enhances minority-class detection, maintains robust predictive performance under concept drift, and significantly outperforms conventional streaming classification methods in terms of efficiency and adaptability. Experimental evaluation demonstrated that the proposed system significantly improves adaptability and classification robustness compared to static feature-based streaming models, particularly in handling feature evolution and imbalanced sentiment distributions in real-time opinion analysis environments.