演講公告
- 更新日期：115-04-27
- 發布單位：工業工程與管理學系
115.5.4〔演講〕以批次估計式為基礎之信賴上界演算法求解具有高斯自相關報酬的多臂老虎機問題
講 題：以批次估計式為基礎之信賴上界演算法求解具有高斯自相關報酬的多臂老虎機問題(Batch-means based upper confidence bound algorithm for multi-armed bandit problems with gaussian correlation rewards)
主 講 人：遲銘璋(Mingchang Chih)(國立中興大學企管系績優副教授兼創新產業暨國際學院企劃行銷組組長)
主 持 人：陳亭志教授
主辦單位：陽明交通大學工業工程與管理系
時間： 115年5月4日(星期一) 13:20 ~15:10
地 點：MB520
演講摘要：This talk focuses on multi-armed bandit problems with temporally dependent Gaussian rewards. In classical bandit settings, reward observations are often assumed to be independent. However, in many real-world applications, rewards exhibit temporal dependence, making conventional algorithms inadequate for fully capturing the uncertainty induced by serial correlation.
To address this issue, our study proposes BM-UCB1-Normal, a batch-means-based extension of UCB1-Normal. The proposed method replaces the conventional variance estimate based on the independent and identically distributed assumption with an estimator that is consistent for the long-run variance (LRV). This design enables a more appropriate balance between exploration and exploitation in weakly stationary dependent reward environments, while preserving the simplicity and effectiveness of upper confidence bound policies.
From a theoretical perspective, this research establishes a finite-time regret upper bound for BM-UCB1-Normal under weakly stationary Gaussian rewards, showing that the regret grows logarithmically with the time horizon. The analysis also highlights the role of batch size and explains how it affects error probabilities and learning performance. In addition, this study develops an information-theoretic framework based on the Kullback–Leibler divergence for dependent Gaussian models. Using AR(1) and MA(1) reward processes as examples, the results show that the long-run variance provides the appropriate scale for characterizing learning efficiency under temporal dependence.
Finally, extensive Monte Carlo experiments support the theoretical findings and demonstrate that BM-UCB1-Normal consistently outperforms the conventional UCB1-Normal in dependent reward settings, with especially notable improvements under strong negative temporal dependence. This talk will present the motivation, methodology, theoretical analysis, and experimental results of the study, and discuss its implications for sequential decision-making in correlated bandit problems.
演講性質：學術研究專題
歡迎聽講
主 講 人：遲銘璋(Mingchang Chih)(國立中興大學企管系績優副教授兼創新產業暨國際學院企劃行銷組組長)
主 持 人：陳亭志教授
主辦單位：陽明交通大學工業工程與管理系
時間： 115年5月4日(星期一) 13:20 ~15:10
地 點：MB520
演講摘要：This talk focuses on multi-armed bandit problems with temporally dependent Gaussian rewards. In classical bandit settings, reward observations are often assumed to be independent. However, in many real-world applications, rewards exhibit temporal dependence, making conventional algorithms inadequate for fully capturing the uncertainty induced by serial correlation.
To address this issue, our study proposes BM-UCB1-Normal, a batch-means-based extension of UCB1-Normal. The proposed method replaces the conventional variance estimate based on the independent and identically distributed assumption with an estimator that is consistent for the long-run variance (LRV). This design enables a more appropriate balance between exploration and exploitation in weakly stationary dependent reward environments, while preserving the simplicity and effectiveness of upper confidence bound policies.
From a theoretical perspective, this research establishes a finite-time regret upper bound for BM-UCB1-Normal under weakly stationary Gaussian rewards, showing that the regret grows logarithmically with the time horizon. The analysis also highlights the role of batch size and explains how it affects error probabilities and learning performance. In addition, this study develops an information-theoretic framework based on the Kullback–Leibler divergence for dependent Gaussian models. Using AR(1) and MA(1) reward processes as examples, the results show that the long-run variance provides the appropriate scale for characterizing learning efficiency under temporal dependence.
Finally, extensive Monte Carlo experiments support the theoretical findings and demonstrate that BM-UCB1-Normal consistently outperforms the conventional UCB1-Normal in dependent reward settings, with especially notable improvements under strong negative temporal dependence. This talk will present the motivation, methodology, theoretical analysis, and experimental results of the study, and discuss its implications for sequential decision-making in correlated bandit problems.
演講性質：學術研究專題
歡迎聽講
