We investigate how to tailor and integrate the deep learning techniques into the conventional financial econometric models for financial time series forecasting. First, a long short-term memory enhanced realized conditional heteroskedasticity model is developed, to explore the full impact of high frequency data based realized volatility on volatility modelling and forecasting via capturing the nonlinear and long-term effects. Further, extending the heterogeneous autoregressive model, a framework known for efficiently capturing long memory in realized measures, a long-memory and non-linear realized volatility model class is proposed for direct Value-at-Risk forecasting by integrating the Recurrent Neural Network. Bayesian inference with Sequential Monte Carlo is employed for model estimation and sequential prediction in the proposed frameworks. Comprehensive empirical study using 31 indices from 2000 to 2022 is conducted. The results demonstrate that our proposed framework achieves superior out-of-sample forecasting performance compared to the benchmark models.