Optimised Support Vector Regression for California Housing Price Prediction: The Critical Role of Feature Engineering and Hyperparameter Tuning

• URL: http://arxiv.org/abs/2605.08660v1
• Date: Sat, 09 May 2026 03:58:50 GMT
• Title: Optimised Support Vector Regression for California Housing Price Prediction: The Critical Role of Feature Engineering and Hyperparameter Tuning
• Abstract summary: Support Vector Regression has been cited as one of the weakest performers on the California Housing benchmark dataset.<n>This paper examines whether the previously reported performance reflects experimental configuration choices rather than an inherent algorithmic limitation.<n> tuned SVR achieves a test R2 of 0.723, a 0.123-point absolute improvement over the previously reported SVR result.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the recent literature, Support Vector Regression (SVR) has been cited as one of the weakest performers on the California Housing benchmark dataset, with Preethi et al. (2025)specifically ranking it last among the algorithms they tested, reporting an R2 of only 0.60. This paper examines whether the previously reported performance reflects experimental configuration choices rather than an inherent algorithmic limitation. A structured experimental workflow is applied: ten domain-motivated derived features are constructed from the eight raw inputs, an exploratory ensemble feature importance analysis identifies the most predictive candidates, and a randomised search over hyperparameter combinations with three-fold cross-validation selects the optimal SVR configuration within a leakage-safe scikit-learn Pipeline. A formal four-stage ablation study isolates the contribution of each component: scaling alone accounts for +0.744 in R2 (from -0.054 to 0.690), feature engineering adds +0.026 (to 0.716), and hyperparameter tuning contributes +0.008 (to 0.723). The resulting tuned SVR achieves a test R2 of 0.723, a 0.123-point absolute improvement over the previously reported SVR result (from 0.60 to 0.723, approximately 20% relative gain). In the ten-model comparison, the tuned SVR ranks fourth with R2 = 0.723, below XGBoost (0.832), Random Forest (0.814) and Gradient Boosting (0.783), while substantially outperforming simpler baselines. Ten-fold cross-validation yields a mean R2 of 0.703 (95% CI: [0.630, 0.775]), confirming robust generalisation. The observed improvement from R2 = 0.60 to R2 = 0.723 is associated primarily with proper feature scaling within a unified preprocessing pipeline, with domain-motivated feature engineering and systematic hyperparameter tuning, providing further incremental gains.

Related papers

• Parameter-Efficient Quantum NLP for Paraphrase Detection: Performance, Robustness, and Entanglement [0.05999777817331314]
  We evaluate a 10-qubit hybrid quantum-classical variational circuit for paraphrase detection.<n>On QQP the circuit achieves 75.53% +/- 0.75% accuracy.<n>On MRPC the optimal 2-layer variant reaches 92% of BERT-base accuracy at 54,000 lower parameter cost.<n>These results constitute the first systematic parameter-matched multi-benchmark empirical validation of hybrid variational circuits for NLP.
  arXiv  Detail & Related papers  (2026-09-13T14:06:49Z)
• EGRL: Edge generation-guided relation-aware learning for RNA-protein interaction prediction [41.17218504310278]
  We propose Edge Generation-guided Relation-aware Learning (EGRL)<n>EGRL is a novel framework with several key components.<n> Comprehensive evaluations on four benchmark datasets demonstrate that EGRL achieves competitive overall performance.
  arXiv  Detail & Related papers  (2026-08-13T07:47:44Z)
• Scaling Closed-Loop Feature Channel Configuration with LLMs [48.83701310501069]
  The analysis covers 2000 generated candidates from 8 complete cycles, yielding 462 verified CIFAR-100 evaluations after task and metadata filtering.<n>The best model reaches 0.3676 with 11.8M parameters, compared with an early high-performing model at 0.3144 with 166.5M parameters.<n>Non-power-of-two channel widths occur in 41.8% of verified candidates, and the strongest models share structured channelallocation patterns.
  arXiv  Detail & Related papers  (2026-07-07T22:31:53Z)
• How Much Capacity Does EEG Denoising Need? Ultra-Compact Networks reveal Benchmark Saturation and Metric-Utility Gap [0.0]
  Deep learning EEG denoising architectures have scaled from tens of thousands to tens of millions of parameters, yet no prior study has isolated model capacity as the experimental variable.<n>We address both gaps by fixing architecture, loss, data split, and training recipe while sweeping only channel width from 1.05K to 40.26K parameters in a minimal depthwiseseparable convolutional U-Net.
  arXiv  Detail & Related papers  (2026-06-07T12:17:25Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment [53.72927532626824]
  We show that subsets of just 50 examples (0.3% of data) can achieve over 0.93 Pearson correlation with full benchmark scores.<n>To better predict preferences, we trained regression models on these selected subsets, achieving 0.98 correlation.<n>This demonstrates that in regression modeling, well-curated subsets outpredict the full benchmark, showing quality over quantity.
  arXiv  Detail & Related papers  (2026-04-20T00:57:31Z)
• REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge [83.2858110368572]
  We propose textbfREAL (underlineREgression-underlineAware Reinforcement underlineLThought), a principled RL framework designed to optimize regression rewards.<n>We show that REAL consistently outperforms both regression-aware SFT baselines and standard RL methods.
  arXiv  Detail & Related papers  (2026-03-17T21:19:08Z)
• Fixed-Budget Parameter-Efficient Training with Frozen Encoders Improves Multimodal Chest X-Ray Classification [0.0]
  Multimodal chest X-Ray analysis often fine-tunes large vision-language models, which is computationally costly.<n>We study parameter-efficient training strategies, including frozen encoders, BitFit, LoRA, and adapters for multi-label classification on the Indiana University Chest X-Ray dataset.
  arXiv  Detail & Related papers  (2025-12-25T05:02:19Z)
• Generalizable Diabetes Risk Stratification via Hybrid Machine Learning Models [0.0]
  Diabetes affects over 537 million people worldwide and is projected to reach 783 million by 2045.<n>We compare two hybrid classifiers and assess their generalizability on an external cohort.
  arXiv  Detail & Related papers  (2025-09-24T21:18:52Z)
• Enhanced Predictive Modeling for Hazardous Near-Earth Object Detection: A Comparative Analysis of Advanced Resampling Strategies and Machine Learning Algorithms in Planetary Risk Assessment [0.0]
  This study evaluates the performance of several machine learning models for predicting hazardous near-Earth objects (NEOs) through a binary classification framework.<n> RFC and GBC performed the best, both with an impressive F2-score of 0.987 and 0.896, respectively.
  arXiv  Detail & Related papers  (2025-08-20T22:50:00Z)
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example [117.86853102104256]
  We show that reinforcement learning with verifiable reward using one training example (1-shot RLVR) is effective in incentivizing the math reasoning capabilities of large language models (LLMs)<n>We identify some interesting phenomena during 1-shot RLVR, including cross-category generalization, increased frequency of self-reflection, and sustained test performance improvement.
  arXiv  Detail & Related papers  (2025-04-29T09:24:30Z)
• Intelligent Gradient Boosting Algorithms for Estimating Strength of Modified Subgrade Soil [0.14843690728081999]
  This study uses categorical boosting (CatBoost) and extreme gradient boosting (XGBoost) and support vector regression (SVR) to estimate properties of subgrade soil modified with hydrated lime activated rice husk ash (HARSH)<n>The results indicate that XGBoost outperformed CatBoost and SVR in estimating these properties, yielding R2 of 0.9994, 0.9995 and 0.9999 in estimating the CBR, UCS and R respectively.
  arXiv  Detail & Related papers  (2025-01-08T20:26:13Z)
• Impact of Comprehensive Data Preprocessing on Predictive Modelling of COVID-19 Mortality [0.0]
  This study evaluates the impact of a custom data preprocessing pipeline on ten machine learning models predicting COVID-19 mortality. Our pipeline differs from a standard preprocessing pipeline through four key steps.
  arXiv  Detail & Related papers  (2024-08-15T13:23:59Z)
• ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning [91.13797346047984]
  We introduce ADAHESSIAN, a second order optimization algorithm which dynamically incorporates the curvature of the loss function via ADAptive estimates. We show that ADAHESSIAN achieves new state-of-the-art results by a large margin as compared to other adaptive optimization methods.
  arXiv  Detail & Related papers  (2020-06-01T05:00:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.