Analysis of Probabilistic Forecasts for NCSW Space Weather Events

LIU Dandan; ZHAO Haijuan; LI Mu; CHEN Anqin; TANG Wei; GUO Jianguang

doi:10.11728/cjss2026.05.2025-0188

Article Contents

Article Navigation > Chinese Journal of Space Science > 2026 > Uncorrected proofOpen Access

LIU Dandan, ZHAO Haijuan, LI Mu, CHEN Anqin, TANG Wei, GUO Jianguang. Analysis of Probabilistic Forecasts for NCSW Space Weather Events (in Chinese). Chinese Journal of Space Science, 2026, 46(5): 1-15 doi: 10.11728/cjss2026.05.2025-0188

Citation:

LIU Dandan, ZHAO Haijuan, LI Mu, CHEN Anqin, TANG Wei, GUO Jianguang. Analysis of Probabilistic Forecasts for NCSW Space Weather Events (in Chinese). Chinese Journal of Space Science, 2026, 46(5): 1-15 doi: 10.11728/cjss2026.05.2025-0188

Citation:

PDF( 7094 KB)

Analysis of Probabilistic Forecasts for NCSW Space Weather Events

doi: 10.11728/cjss2026.05.2025-0188 cstr: 32142.14.cjss.2025-0188

National Satellite Meteorological Center (National Center for Space Weather), China Meteorological Administration, Beijing 100081
Innovation Center for FengYun Meteorological Satellite (FYSIC), China Meteorological Administration, Beijing 100081
Key Laboratory of Space Weather, China Meteorological Administration, Beijing 100081

Received Date: 2025-11-10
Rev Recd Date: 2026-07-23

Available Online: 2026-07-26

Abstract

Abstract

Based on the probabilistic forecast data of space weather events from the National Center for Space Weather, China (NCSW) during 2005–2022, this study systematically examines the forecast performance of M-class flares, X-class flares, proton events, minor geomagnetic storms, and moderate-and-above geomagnetic storms, and compares the results with the U.S. Space Weather Prediction Center (SWPC) within the reliability-sharpness-resolution framework. The results show that the forecast skill of all event types degrades with increasing lead time. For M-class flares, the forecast performance is stable in years with sufficient samples but fluctuates obviously during solar minimum. X-class flares have scarce samples and prominent false alarms, with an operational strategy of “preferring false alarms to misses”. Proton event forecasts adopt a conservative zero-false-alarm strategy. Under comparable sample conditions, the forecast skill of minor geomagnetic storms improves significantly in later periods compared with earlier stages. Moderate-and-above geomagnetic storms have low occurrence frequency and generally weak forecast skill. Multi-metric joint analysis indicates that TSS is suitable for annual operational capability tracking, and statistical characteristic analysis serves long-term systematic calibration. The PC metric is strongly affected by the event base rate and requires comprehensive assessment combined with TSS. The comparison shows that NCSW has advantages in forecast reliability and false alarm control, while SWPC achieves stronger sharpness. NCSW presents better overall discrimination skill at short lead times, and the performance difference between the two agencies gradually narrows with extended lead time. This study conducts continuous verification of NCSW’s 18-year probabilistic forecasts and reveals the forecast characteristics of rare space weather events, which provides support for China’s space weather operational assessment and forecasting capability development.
- Space weather events,
- Solar activity level,
- Solar flares,
- Solar proton events,
- Geomagnetic storms,
- TS score,
- TSS score

FullText(HTML)

References(36)

References

[1]	汪景琇. 太阳剧烈活动与空间灾害天气[J]. 紫金山天文台台刊, 2003, 22(1): 48-57 WANG Jingxiu. Explosive solar activity and disastrous space weather[J]. Publications of Purple Mountain Observatory, 2003, 22(1): 48-57
[2]	王胜国, 谢志辉, 赵增亮, 等. 灾害性空间天气对国家安全的影响与对策[C]//国家安全地球物理丛书(九)——防灾减灾与国家安全. 西安: 西安地图出版社, 2013: 37-42 WANG Shengguo, XIE Zhihui, ZHAO Zengliang, et al. Influence on national security of disastrous space weather[C]//National Security Geophysics Series (9): Disaster Prevention and Reduction and National Security. Xi’an: Xi’an Map Publishing House, 2013: 37-42
[3]	李小明. 空间天气电网磁暴灾害监测系统的研究与实现[D]. 保定: 华北电力大学, 2013 LI Xiaoming. Research and Implement of Space Weather Power Grid Geomagnetic Storm Monitoring System[D]. Baoding: North China Electric Power University, 2013
[4]	SCHRIJVER C J, KAURISTIE K, AYLWARD A D, et al. Understanding space weather to shield society: a global road map for 2015-2025 commissioned by COSPAR and ILWS[J]. Advances in Space Research, 2015, 55(12): 2745-2807 doi: 10.1016/j.asr.2015.03.023
[5]	ZHENG Y H, GANUSHKINA N Y, JIGGENS P, et al. Space radiation and plasma effects on satellites and aviation: quantities and metrics for tracking performance of space weather environment models[J]. Space Weather, 2019, 17(10): 1384-1403 doi: 10.1029/2018SW002042
[6]	王劲松, 焦维新. 空间天气灾害[M]. 北京: 气象出版社, 2009 WANG Jinsong, JIAO Weixin. Space Weather Disasters[M]. Beijing: China Meteorological Press, 2009 (in Chinese)
[7]	CALDWELL B, MCCARRON E, JONAS S. An abridged history of federal involvement in space weather forecasting[J]. Space Weather, 2017, 15(10): 1222-1237 doi: 10.1002/2017SW001626
[8]	LILENSTEN J, BELEHAKI A. Developing the scientific basis for monitoring, modelling and predicting space weather[J]. Acta Geophysica, 2009, 57(1): 1-14 doi: 10.2478/s11600-008-0081-3
[9]	王劲松. 中国气象局的空间天气业务[J]. 气象科技进展, 2011, 1(4): 6-12 WANG Jinsong. Space weather operations in CMA[J]. Advances in Meteorological Science and Technology, 2011, 1(4): 6-12
[10]	LIU S Q, GONG J C. Operational space weather services in national space science center of Chinese academy of sciences[J]. Space Weather, 2015, 13(10): 599-605 doi: 10.1002/2015SW001298
[11]	牟惟丰. 预报评分方法述评和方案建议[J]. 气象, 1986, 12(2): 45-49 MOU Weifeng. Review of forecast verification methods and proposals for schemes[J]. Meteorological Monthly, 1986, 12(2): 45-49
[12]	CROWN M D. Validation of the NOAA Space Weather Prediction Center’s solar flare forecasting look-up table and forecaster-issued probabilities[J]. Space Weather, 2012, 10(6): 2011SW000760 doi: 10.1029/2011SW000760
[13]	CAMPOREALE E, BERGER T E. Verification of the NOAA Space Weather Prediction Center solar flare forecast (1998-2024)[J]. Space Weather, 2025, 23(10): e2025SW004546 doi: 10.1029/2025SW004546
[14]	BALCH C C. Updated verification of the Space Weather Prediction Center’s solar energetic particle prediction model[J]. Space Weather, 2008, 6(1): 2007SW000337. doi: 10.1029/2007SW000337
[15]	BAIN H M, STEENBURGH R A, ONSAGER T G, et al. A summary of National Oceanic and Atmospheric Administration Space Weather Prediction Center proton event forecast performance and skill[J]. Space Weather, 2021, 19(7): e2020SW002670. doi: 10.1029/2020SW002670
[16]	SHARPE M A, MURRAY S A. Verification of space weather forecasts issued by the Met Office Space Weather Operations Centre[J]. Space Weather, 2017, 15(10): 1383-1395 doi: 10.1002/2017SW001683
[17]	MURRAY S A, BINGHAM S, SHARPE M, et al. Flare forecasting at the Met Office Space Weather Operations Centre[J]. Space Weather, 2017, 15(4): 577-588 doi: 10.1002/2016SW001579
[18]	DEVOS A, VERBEECK C, ROBBRECHT E. Verification of space weather forecasting at the Regional Warning Center in Belgium[J]. Journal of Space Weather and Space Climate, 2014, 4: A29 doi: 10.1051/swsc/2014025
[19]	CUI Y M, LIU S Q, ERCHA A, et al. Verification of SPE probability forecasts at the Space Environment Prediction Center (SEPC)[J]. Science China Earth Sciences, 2016, 59(6): 1292-1298 doi: 10.1007/s11430-016-5284-x
[20]	刘丹丹, 赵海娟, 唐伟. 地磁暴短期概率预报水平的检验评估[J]. 地球物理学报, 2026, 69(7): 2843-2857 doi: 10.6038/cjg2026T0007 LIU Dandan, ZHAO Haijuan, TANG Wei. Verification of short-term geomagnetic storm forecasts[J]. Chinese Journal of Geophysics, 2026, 69(7): 2843-2857 doi: 10.6038/cjg2026T0007
[21]	林元章. 太阳物理导论[M]. 北京: 科学出版社, 2000 LIN Yuanzhang. Introduction to Solar Physics[M]. Beijing: Science Press, 2000 (in Chinese)
[22]	全国卫星气象与空间天气标准化委员会空间天气监测预警分技术委员会. 太阳软X射线耀斑强度分级: GB/T 31157-2014[S]. 北京: 中国标准出版社, 2014: 9
[23]	涂传诒, 宗秋刚, 何建森等. 日地空间物理学(第二版)(上册): 日球层物理[M]. 北京: 科学出版社, 2020 TU Chuanyi, Zong Qiugang, He Jiansen, et al. Space Physics (2nd Edition, Vol. 1): Heliospheric Physics[M]. Beijing: Science Press, 2020
[24]	ONSAGER T, GRUBB R, KUNCHES J, et al. Operational uses of the GOES energetic particle detectors[C]// Proceedings Volume 2812, GOES-8 and Beyond. Denver: International Society for Optics and Photonics, 1996, 2812: 281-290
[25]	WILKS D S. Statistical Methods in the Atmospheric Sciences[M]. 2nd ed. Amsterdam: Elsevier, 2006
[26]	涂传诒, 宗秋刚, 周煦之. 日地空间物理学(行星际与磁层)(下册)[M]. 北京: 科学出版社, 2020 TU Chuanyi, ZONG Qiugang, ZHOU Xuzhi. Space Physics (Interplanetary and Magnetosphere) (Vol. 2)[M]. Beijing: Science Press, 2020
[27]	全国卫星气象与空间天气标准化委员会空间天气监测预警分技术委员会. 空间天气短期预报检验方法: QX/T 295-2015[S]. 北京: 中国气象出版社, 2015
[28]	JOLLIFFE I T, STEPHENSON D B. Forecast Verification: A Practitioner’s Guide In Atmospheric Science[M]. 2nd ed. Chichester: Wiley Blackwell, 2012: 274
[29]	FAWCETT T. An introduction to ROC analysis[J]. Pattern Recognition Letters, 2006, 27(8): 861-874 doi: 10.1016/j.patrec.2005.10.010
[30]	GONZALEZ W D, JOSELYN J A, KAMIDE Y, et al. What is a geomagnetic storm?[J]. Journal of Geophysical Research: Space Physics, 1994, 99(A4): 5771-5792 doi: 10.1029/93JA02867
[31]	RICHARDSON I G, CANE H V. Near-Earth interplanetary coronal mass ejections during solar cycle 23 (1996-2009): Catalog and summary of properties[J]. Solar Physics, 2010, 264(1): 189-237 doi: 10.1007/s11207-010-9568-6
[32]	KILPUA E K J, HIETALA H, TURNER D L, et al. Unraveling the drivers of the storm time radiation belt response[J]. Geophysical Research Letters, 2015, 42(9): 3076-3084 doi: 10.1002/2015GL063542
[33]	HATHAWAY D H. The solar cycle[J]. Living Reviews in Solar Physics, 2015, 12(1): 4 doi: 10.1007/lrsp-2015-4
[34]	BLOOMFIELD D S, HIGGINS P A, MCATEER R T J, et al. Toward reliable benchmarking of solar flare forecasting methods[J]. The Astrophysical Journal Letters, 2012, 747(2): L41 doi: 10.1088/2041-8205/747/2/L41
[35]	DOSWELL III C A, DAVIES-JONES R, KELLER D L. On summary measures of skill in rare event forecasting based on contingency tables[J]. Weather and Forecasting, 1990, 5(4): 576-585 doi: 10.1175/1520-0434(1991)006<0400:cosmos>2.0.co;2
[36]	FERRO C A T, FRICKER T E. A bias-corrected decomposition of the Brier score[J]. Quarterly Journal of the Royal Meteorological Society, 2012, 138(668): 1954-1960 doi: 10.1002/qj.1924