Project Field : Process Control
Project Title : Optimisation and Control of Fed-Batch Yeast Production using Q-Learning
Project : MEng in Chemical Engineering
Project Duration : July 2010 – Oct 2012
In this work, the optimal production of yeast with minimal production of ethanol in fed-batch yeast fermentation is investigated. Q-learning (QL) is a heuristic approach suggested for the process dynamic handling to achieve the multiobjectives optimisation. The QL agent interacts with the fermentation environment will gain experience on the state transitions, which are represented by the change of substrate, yeast, oxygen and ethanol concentration and the system volume. In the present study, multistep action (MSA) has been implemented in consideration of the inborn process delay for the substrate feeding to take effect on yeast growth. Parameter deviated model has been implemented in the QL to test the robustness of the algorithm besides to identify the process disturbance. Q-learning with exploration (QLE) shows the importance of exploration based on the developed Q-table to optimise the process. From the result, QL was able to perform multiobjectives decision making for the optimal substrate feeding profile. The final yeast production using QL-optimised feeding profile is 20.86% higher compare to the nominal exponential feeding (EF), and 19.59% higher compare to EF with process disturbance. On the other hand, the performance of QLE is also tested for both nominal and disturbance case and yielded 51.00% and 46.87% higher yeast production than EF respectively, while maintaining low ethanol production. In a nutshell, QL is an alternative that can be considered to perform multiobjective optimisation in a frequently changing bioenvironment and suggest a substrate feeding profile that satisfied the process goal. The QLE can cope better with the process disturbance.
Contact Person : Helen Chuo Sin Ee
Contact Email : hsechuo@gmail.com
Publications :
International Journals
2013
[J1301] H.S.E. Chuo, M.K. Tan, H.J. Tham, K.T.K. Teo. Q-Learning based Controller for Fed-Batch Yeast Fermentation. Developments in Sustainable Chemical and Bioprocess Technology, DSCBT. ISBN: 978-1-4614-6208-8. September 2013. pp.219-225. New York: Springer US. [pdf] [published]
doi: 10.1007/978-1-4614-6208-8_28 [Scopus]
2012
[J1201] M.K. Tan, H.S.E., Chuo, Y.Y. Farm, H.J. Tham, K.T.K. Teo. Control of Exothermic Batch Process using Multivariable Genetic Algorithm. International Journal of Simulation – Systems, Science & Technology, IJSSST. ISSN: 1473-804x. June 2012. vol. 13, no. 3C, pp. 19-25. [pdf][published]
International Proceedings
2012
[P1201] M.K. Tan, H.S.E. Chuo, H.J. Tham, Y.Y. Farm, K.T.K. Teo. Genetic Algorithm based Multivariable Control for Exothermic Batch Process. 4th International Conference on Computational Intelligence, Communication Systems and Networks [Phuket, Thailand]. Proceeding of CICSyN 2012. ISBN: 978-0-7695-4821-0. 24-26 July 2012. pp. 32-37. [pdf] [published]
doi: 10.1109/CICSyN.2012.17 [IEEE Xplore, Scopus]
[P1202] H.S.E. Chuo, M.K. Tan, H.J. Tham, K.T.K. Teo. Q-Learning based Proportional Controller for Fed-Batch Yeast Fermentation. 26th Symposium of Malaysian Chemical Engineers [Kota Kinabalu, Malaysia]. Proceeding of SOMChE 2012. 21-23 November 2012. [published]
[P1203] M.K. Tan, H.S.E. Chuo, H.J. Tham, K.T.K. Teo. Exothermic Batch Process Optimisation via Mulitvariable Genetic Algorithm. International Conference on Advanced Computer Science Applications and Technologies 2012 [Kuala Lumpur, Malaysia]. Proceeding of ACSAT 2012. 26-28 November 2012. [published]
2011
[P1101] H.S.E. Chuo, M.K. Tan, H.J. Tham, K.T.K. Teo. Optimization of Fed-Batch Baker’s Yeast Fermentation Process using Learning Algorithm. 27-28 October 2011, 18th Regional Symposium on Chemical Engineering [Ho Chi Minh City, Vietnam]. Proceeding of RSCE 2011. [URL] [published]
[P1102] H.S.E. Chuo, M.K. Tan, H.J. Tham, K.T.K. Teo. Comparative Study on the Application of Q-learning in Fed-batch Bioprocess. 28 November – 1 December 2011, International Conference of Chemical Engineering and Industrial Biotechnology [Pahang, Malaysia]. Proceeding of SOMChE 2011. [URL] [published]
Modelling, Simulation & Computing Laboratory (mscLab) 