{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sharing-deep-generative-representation-for","title":"Sharing deep generative representation for perceived image reconstruction from human brain activity","arxiv_id":"1704.07575","date":"2017-04-25","proceeding":null,"authors":["Changde Du","Changying Du","Huiguang He"],"abstract":"Decoding human brain activities via functional magnetic resonance imaging\n(fMRI) has gained increasing attention in recent years. While encouraging\nresults have been reported in brain states classification tasks, reconstructing\nthe details of human visual experience still remains difficult. Two main\nchallenges that hinder the development of effective models are the perplexing\nfMRI measurement noise and the high dimensionality of limited data instances.\nExisting methods generally suffer from one or both of these issues and yield\ndissatisfactory results. In this paper, we tackle this problem by casting the\nreconstruction of visual stimulus as the Bayesian inference of missing view in\na multiview latent variable model. Sharing a common latent representation, our\njoint generative model of external stimulus and brain response is not only\n\"deep\" in extracting nonlinear features from visual images, but also powerful\nin capturing correlations among voxel activities of fMRI recordings. The\nnonlinearity and deep structure endow our model with strong representation\nability, while the correlations of voxel activities are critical for\nsuppressing noise and improving prediction. We devise an efficient variational\nBayesian method to infer the latent variables and the model parameters. To\nfurther improve the reconstruction accuracy, the latent representations of\ntesting instances are enforced to be close to that of their neighbours from the\ntraining set via posterior regularization. Experiments on three fMRI recording\ndatasets demonstrate that our approach can more accurately reconstruct visual\nstimuli.","url_abs":"http://arxiv.org/abs/1704.07575v3","url_pdf":"http://arxiv.org/pdf/1704.07575v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sharing-deep-generative-representation-for","repo_url":"https://github.com/ChangdeDu/DGMM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"image-reconstruction","task_name":"Image Reconstruction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1704.07575","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}