Professor Jiebo Luo Albert Arendt Hopeman Professor Department of Computer Science 3101 Wegmans Hall  University of Rochester Rochester, NY 14627 Email: username -at- domain.edu where username = jluo, domain = cs.rochester.edu Phone: 585-276-5784 Prospective Students

Computer Vision Urban Computing Health/Wellness Analytics Image Retrieval Image Segmentation Scene Classification Big Data Contextual Inference Machine Learning  Vision+Language Random Field Photo Collection Multimedia    Video Analytics Human-Centered Media & Interaction Mobile Media Location-based service Geo Tagging  Action Recognition Heterogeneous Information Networks Social Multimedia  Data Mining Crowdsourcing 360D User Profiling

News

    2024

• 11/04/2024: Prof. Luo is elected to Fellow of AIMBE (American Institute for Medical and Biological Engineering)
  
• 10/30/2024: Prof. Luo is a Keynote Speaker at the 2024 ACM Multimedia Conference
• 10/16/2024: Prof. Luo wins the inaugural Debra Haring Excellence in Research Award, which recognizes a faculty member who has done research that has had a transformational impact on the field, or who has accumulated a body of research that has changed the field in positive ways.
  
• 10/10/2024: Some members of VIStA watched the magnificent aurora together off Webster Pier in Rochester, NY
  

• 09/12/2024: Prof. Luo is an invited speaker at the Global AI Summit (GAIN)
• 08/29/2024: Congratulations to Hanjia Lyu for being awarded the 2024 Google PhD Fellowship 

• 08/27/2024: IEEE CS releases an interview with Prof. Luo at CVPR 2024 on industry vs. academic careers, why publishing, why attending conferences, how to choose research topics, and so on.
  
• 08/07/2024: Our study that shows the tech community is split over AI is covered by Futurity, Quantum Zeitgeist, and CODU Compute. 
  
• 05/18/2024: Our study on the factors contributing to crowdfunding success is featured in the Wall Street Journal
   
  
• 05/15/2024: Prof. Luo receives another major award - the 2024 Edmund A. Hajim Outstanding Faculty Award
  
• 04/08/2024: (Day of Total Solar Eclipse): Prof. Luo receives the 2024 William H. Riker University Award for Excellence in Graduate Teaching, which recognizes a faculty member who has excelled in graduate instruction, particularly in the University's doctoral programs.

• 03/28/2024: Congratulations to Alexander Martin for winning the 2024 Charles L. Newton Prize (awarded to senior engineering students who show a special proficiency in an engineering subject and have conducted research, given a presentation, or published a paper)
  
• 01/15/2024: Congratulations to Alexander Martin for an Honorable Mention in the 2023-2024 Computing Research Association's Outstanding Undergraduate Researcher Award Competition
  
• 01/04/2024: We release a joint survey of Video Understanding with Large Language Models

    2023

• 12/12/2023: We release a joint survey of Large Language Models in Medicine: Principles, Applications, and Challenges
  

• 11/15/2023: We release an evaluation report on GPT-4V(ision) as A Social Media Analysis Engine

• 10/25/2023: Professor Luo serves as General Co-Chair of IEEE ICME 2024 (Call for Papers) in Niagara Falls
  

• 09/18/2023: News 10 NBC WHEC-TV Rochester features our NIH and NSF funded project SMARTeeth in their In-Depth Investigative program entitled "Bringing the dentist to you: UR developing app to identify tooth decay in children"; also reported by News 8 WROC-TV in their Rochester First program entitled "Eastman Institute using AI creates app for early tooth decay detection"

• 09/13/2023: We receive the Adobe 2023 Data Science Research Award to support our data science research
  

• 07/13/2023: In our new studies (ICWSM 2024, ICWSMW 2023), we uncover media bias in 1.8 million headlines across the political spectrum. Our findings suggest that media bias is growing. The work is covered by StudyFinds, and shared by ScienceBlog, RealClearScience, Nation, Mirage.News, Phys.org, and the American Association for the Advancement of Science (AAAS) via EurekAlert!

• 07/10/2023: Congratulations to Patrick Jingyuan Chen (a rising junior) for winning the Best Student Paper Award at the 2023 IEEE International Conference on Digital Health (ICDH) for the work on "sDREAMER: Self-distilled Hierarchical Mixture of Modality Experts Transformer are Efficient Mice Sleep Learners"
• 06/29/2023: Congrats to newly minted Dr. Songyang Zhang who is joining Amazon Research

• 06/17/2023: Professor Luo receives the 2023 Contribution Award from Health Data Science (a Science partner journal) "in recognition of contributions to Health Data Science"
  
• 05/22/2023: Congrats to newly minted Dr. Wei Zhu who has joined Amazon Research
• 05/19/2023: Congrats to newly minted Dr. Haitian Zheng who is joining Adobe Research

• 05/12/2023: Congratulations to Peirong Hao for receiving the CS Department Best Senior Award
  

• 05/12/2023: Congratulations to Yufei Zhao for receiving the CS Excellence in Undergraduate Research Award and Donald M. Barnard Prize (awarded to engineering students on the basis of personal qualification and achievement)
  

• 03/09/2023: Congratulations to Hanjia Lyu for being awarded the 2023 Arts, Sciences and Engineering Donald M. and Janet C. Barnard Fellowship

    2022

• 12/08/2022: Professor Luo is elected to Fellow of NAI (National Academy of Inventors) for "have demonstrated a highly prolific spirit of innovation in creating or facilitating outstanding inventions that have made a tangible impact on the quality of life, economic development, and welfare of society"

• 10/10/2022: Congrats to Zhengyuan Yang, who graduated in 2021 from URCS and is now a Senior Researcher with Microsoft Research, for winning the 2022 ACM SIGMM Award for Outstanding PhD Thesis for his PhD thesis entitled "Visual Grounding: Building Cross-Modal Visual-Text Alignment"

• 09/01/2022: Professor Luo is recognized as the Albert Arendt Hopeman Professor of Engineering

• 08/01/2022: Professor Luo's group joins the Data Score of the NIH U19 project on Neural Circuit Control of Fluid and Solute Clearance During Sleep (by UR, MIT, and UPenn under the NIH Brain Initiative)
  

• 06/17/2022: Professor Luo is elected to a Member of Academia Europaea, a pan-European academy of humanities, law, and science based on sustained academic excellence in his field, as well as significant Europe-wide activities
• 06/14/2022: Congrats to newly minted Dr. Wei Xiong who is joining Adobe Photoshop
• 06/09/2022: Congrats to newly minted Dr. Weijian Li who is joining Amazon Go
  

• 03/25/2022: Our study on the scale and scope of the influence of misinformation and fact-based news about COVID-19 vaccines on social media platforms on the vaccine uptake is covered by multiple news outlets including Medical Economics, News Medical, News Wise, UK Today News, and Medical Xpress
  
• 02/01/2022: Application is open for NSF Summer REU on Imaging in Medicine and Biology for Underrepresented  Minorities at University of Rochester

    2021

• 10/22/2021: Professor Luo receives the 2021 ACM SIGMM Award for Outstanding Technical Contributions to Multimedia Computing, Communications and Applications for his "outstanding, pioneering and continued research contributions in the areas of multimedia content analysis and social media analytics and for outstanding and continued service to the multimedia community"
  

• 09/21/2021: Professor Luo reaches a milestone of 100 in his h-index according to Google Scholar

• 07/08/2021: Professor Luo delivers a keynote on "Vision and Language: The Past, Present and Future" at the 2021 IEEE International Conference on Multimedia and Expo (ICME). The slides are available in slideshare.net.
• 06/25/2021: Congrats to newly minted Dr. Tianlang Chen who has joined Amazon
• 06/07/2021: Congrats to newly minted Dr. Zhengyuan Yang who has joined Microsoft Research

• 05/26/2021: Congratulations to Songyang Zhang for winning the NAACL Best Long Paper Award at the 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) for the work on "Video-aided Unsupervised Grammar Induction"

• 05/19/2021: In a new study, we dissect the public responses to the #StopAsianHate movement using large scale Twitter data and provide findings that can help design better ways of reducing tension and misunderstandings between ethnic groups. The work is also covered by Rochester Beacon and Futurity.
• 01/29/2021: U.S. Senate Majority Leader Charles E. Schumer and U.S. Senator Kirsten Gillibrand today announced federal funding to support the research and development of oral disease detection and intervention technology at the University of Rochester

    2020

• 12/15/2020: Congratulations to Yipeng Zhang, who is selected for Honorable Mention for the Computing Research Association's (CRA) Outstanding Undergraduate Researcher Award for 2021

• 12/12/2020: The latest CS Rankings data show that in the area of Computer Vision, Rochester ranks 8th in the United States for the year of 2020
• 11/27/2020: Data Skeptic, a popular data science podcast reaching 30k+ listeners nationwide per week, airs a full episode on our recent work on the U.S. facemask sentiment analysis using Twitter data. The recording is available online and via Spotify.          
• 07/23/2020: Tech Xplore reports a series of our recent studies that show Twitter mirrors our attitudes and feelings about COVID-19. These findings are shared in a lightning presentation for chairs of computer science PhD-granting departments and heads of major industrial research labs at the Computing Research Association (CRA) Virtual Conference.
• 05/30/2020: Our work on analyzing the use of controversial terms of COVID-19 on Twitter is highlighted by IEEE Spectrum, the flagship magazine and website of the IEEE. Incidentally, this story shares the headlines with the story about the successful launch of the SpaceX Dragon 2 spacecraft. Cool!
• 04/27/2020: Congrats to newly minted Dr. Yang Feng who will join Google Cloud

• 01/01/2020: Professor Luo begins his 3-year term as the Editor-in-Chief of the IEEE Transactions on Multimedia

    2019

• 12/31/2019: The latest CS Rankings data show that in the area of Computer Vision, Rochester ranks in the Top 5  in the United States for the year of 2019, trailing only CMU, Stanford, UCFand Berkeley, and right ahead of MIT and Cornell

• 12/12/2019: Congratulations to Zhengyuan Yang for winning the 2020 Twitch Research Fellowship to support his PhD research on video and language. Twitch is the No. 1 live streaming video service for video games in the US, operated by Twitch Interactive, a subsidiary of Amazon. In addition, it offers music broadcasts, creative content, and more recently, "in real life" streams. 
• 11/08/2019: Congrats to newly minted Dr. Haofu Liao who will join Amazon (AWS)
  
• 10/23/2019: Congrats to Songyang Zhang for winning the 1st place in the HACS Action Localization Challenge at ICCV 2019
• 10/17/2019: Our team wins the 1st place in the Challenge on Automatic Generation of Cardiovascular Diagnostic Report (AutoGen-CDR19) at MICCAI 2019
• 10/08/2019: Congratulations to Haofu Liao for being selected as a finalist for MICCAI 2019 Young Scientist Award based his work on Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction 
  
• 09/13/2019: Congrats to newly minted Dr. Jianbo Yuan who will join Amazon (AWS)
• 08/08/2019: Congratulations to Weijian Li for being selected for a graduate student travel award on the basis of the score obtained during the review process. Weijian will present papers at MICCAI 2019. Yipeng Zhang also receives an undergraduate student travel award.
  
• 07/29/2019: Yipeng Zhang, a CS sophomore and a Xerox undergraduate engineering fellow in this summer, is featured in an Instagram post by our new University President Sarah C. Mangelsdorf. It includes a picture of Yipeng
  with Sarah in front of Yipeng's poster on "A deep learning approach to early childhood caries prevention.
• 06/18/2019: Congratulations to Yang Feng who participates in the CVPR 2019 Doctoral Consortium
• 04/03/2019: Congrats to newly minted Dr. Tianran Hu who will join College of William and Mary as Assistant Professor of Computer Science

    2018

• 12/07/2018: Professor Luo is elevated to Fellow of AAAI (Association for Advancement of Artificial Intelligence) for "significant contributions to the fields of computer vision and data mining, and particularly pioneering work                 on multimodal understanding for sentiment analysis, computational social science, and digital health"
  
• 11/01/2018: Professor Luo is elevated to Fellow of ACM (Association for Computing Machinery) for "contributions to multimedia content analysis and social multimedia informatics"
  

• 10/03/2018: The University of Rochester has been selected as a Morris K. Udall Center of Excellence in Parkinson's Disease Research by the National Institute of Neurological Disorders and Stroke (NINDS). The new $9.2 million award brings together researchers from industry and multiple academic institutions to focus on the development of digital tools to enhance understanding of the disease, engage broad populations in research, and accelerate the development of new treatments for Parkinson's disease. Professor Luo will co-lead the Advanced Analytics Core that brings together a team of experts in biostatistics, computational science and mathematics from multiple institutions.


• 09/15/2018: Professor Luo is the recipient of the 2018 IEEE Region 1 Technological Innovation in Academic Award for "contributions in computer vision and data mining"


• 08/24/2018: Congratulations to Zhengyuan Yang and Yixuan Zhang for winning Best Industrial Related Paper Award at ICPR 2018 for their paper on "End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perception" 
• 08/13/2018: Congrats to newly minted Dr. Yu Wang  who has joined Apple
• 08/06/2018: Congratulations to Tianran Hu for being invited to the 2018 ACM Multimedia Doctoral Symposium
  
• 06/29/2018: Congratulations to Haofu Liao for being "pre-selected for a student travel award on the basis of the score your paper has obtained during the review process". Haofu will present two papers (both with early acceptance) at MICCAI 2018.

    2017

• 06/22/2017: Professor Luo is an invited speaker at CVPR 2018 Workshop on Towards Automatic Understanding of Visual Advertisements
  
• 06/21/2017: Professor Luo hosts the 3rd annual "The Vision Seeker" (视觉求索) gathering at CVPR 2018
  

                   
                    Scan QR Code via WeChat 
                    to follow Official Account

• 02/13/2017: Professor Luo and multidisciplinary team are featured in a report by the University of Rochester Newscenter, "App combines virtual reality with personalized psychotherapy"
• 12/25/2017: Professor Luo is interviewed by Sixthtone in a feature "Even China's Backwater Cities Are Going Smart" which reveals China's grand plans for an artificial intelligence rollout of smart cities.
• 11/22/2017: Prof. Homer Chen visits our lab
• 10/13/2017: Congrats to newly minted Dr. Yuncheng Li  who has joined Snapchat Research
• 08/29/2017: Congrats to newly minted Dr. Quanzeng You  who is joining Microsoft Research AI


• 08/21/2017: Professor Luo is quoted by Market Mogul  in a feature "Here's How Big Data Transformation Will Sweep Information Sovereignty"
• 08/14/2017: Professor Luo is interviewed by Bloomberg Technology in a feature"China's Plan for World Domination in AI Isn't So Crazy After All"
• 07/26/2017:  Professor Luo is an invited speaker at CVPR 2017 Workshop on Medical Computer Vision and Health Informatics

• 07/21/2017:  Professor Luo is co-organizing Language and Vision Workshop at CVPR 2017  
• 07/21/2017:  Professor Luo is co-teaching CVPR 2017 Tutorial (lecture notes online) on Dealing with Reality: Low-Quality Visual Data Processing and Analytics
• 05/29/2017:  Professor Luo is interviewed by Xinhua News Agency on the heels of AlphaGo's triumph over Ke Jie Spotlight: Artificial intelligence fuels visions of how smart future can be
• 03/20/2017:  Our work on Detecting the Hate Code on Social Media is covered extensively by Geek.com, Vice.com, Digital Trends, MSN, Thestack.com Our work is especially timely given that 1) YouTube is facing a full-scale advertising boycott over hate speech, and 2) Tech Giants Vow to Tackle Online Hate Speech Within 24 Hours

• 02/24/2017:  Professor Luo's Data Mining course was considered "favorite class taken at the University of Rochester" in the Data Science program (Featured Article: Data science for a better planet)
• 02/11/2017:  National Science Foundation Research Experience for Undergraduates (NSF REU) Computational Methods for Understanding Music, Media, and Minds. Please note that the projects are flexible and the current descriptions are meant as examples.

    2016

• 11/07/2016: Professor Luo is a panelist on NPR Radio, WXXI Connections hosted by Evan Dawson Monthly Science Roundtable "Twitter, Social Media and The Election"
• 10/21/2016: Our work on fine-grained recognition of East Asian faces is covered by Washington Post. You are welcome to take the Asian Face Challenge at http://www.vista-today.com/asianmyth to help benchmark human recognition performance (currently at 49% with 400+ tests). Another story on NextShark

• 10/01/2016: Professor Luo speaks at the Huang Symposium @UIUC to celebrate the 80th birthday and  extraordinary career of Professor Thomas S. Huang

• 08/29/2016: Congratulations to Tianran Hu who receives a student travel scholarship to attend CIKM 2016.
• 07/20/2016: URCS is highly ranked among the Top universities in Artificial Intelligence for the period of 2000-2016 by an objective measure of faculty publications
• 07/19/2016: URCS is among the Top 20 Artificial Intelligence Engineering Schools in the U.S.

• 07/01/2016: Congratulations to Quanzeng You for being invited to the 2016 ACM Multimedia Doctoral Symposium with a travel scholarship
• 06/26/2016: Yiheng Zhou and Numair Sani, our undergraduate researchers, present their work on mining Instagram for illicit drug use patterns at SBP-BRiMS 2016, supported by student travel scholarship
• 06/09/2016: Our work on "Will Sanders Supporters Jump Ship to Trump?" is covered by Pacific Standard
• 05/10/2016: Our work on using AI to hunt drug dealers on Instagram is covered by Daily Dot
• 05/04/2016: Our work on "Voting with Feet: Who are leaving Hilary Clinton and Donald Trump"  is covered by AOL, Daily Dot, LipTV, Pivot America
• 04/21/2016: Zhishen Pan, a Junior student in the Data Science program, won the Poster Presentation Award at the 2nd Annual Rochester Global Health Symposium & UNYTE Scientific Session: Innovative Solutions to Combat Global Health Disparities for the poster "Towards Understanding How News Coverage Affects Public Perception During Epidemic Outbreak."  Congratulations!
• 04/18/2016: Yu Wang receives a student travel scholarship to attend ICWSM-2016

• 03/29/2016: Congratulations to Yuncheng Li, who has been invited to the CVPR 2016 Doctoral Consortium
• 03/20/2016: Our work on using Twitter user avatar facial analysis to study Trump and Clinton followers is covered by MIT Technology Review, Digital Trends, Market Exclusive
• 03/17/2016: Our CVPR 2016 paper -  "Image Captioning with Semantic Attention" - describes the engine behind a system that has been sitting atop the Leader Board for the Microsoft COCO (Common Objects in Context) Image Captioning Challenge since November 2015, beating Google, Microsoft, Baidu/UCLA, Stanford, UC Berkeley, University of Toronto/Montreal, and other power houses. Full Story
• 03/16/2016: Our work on detecting drinking-while-tweeting patterns is covered by MIT Technology Review,  Gizmodo, CBS News, NPR Weekend Edition, Engadget, and more
• 02/06/2016: Our work on estimating city air pollution from crowd-sourced photos is covered by New Scientist Try our demo system

    2015

• 12/21/2015:  Quanzeng You is a Finalist for the inaugural Adobe Research Fellowship
• 11/01/2015: Our work on using Instagram posts to determine underage drinking patterns has been covered by Innovation Trail, Digital Trends, Democratic and Chronicle, TechRadar, Medical Daily, The Science Explorer, iTech Post
• 10/01/2015: Yu Wang and Tianran Hu receive student travel scholarships to attend IEEE ICDM 2015
• 10/02/2015: Data Science Faculty Search (2015-2016): neurally-inspired computing; computer vision for medicine and humanities
• 10/01/2015: Yu Wang receives a student travel scholarship to attend IEEE BigData 2015
•  08/28/2015: Our data-driven study on the influence of fashion shows on everyday trends has been featured by New York Post, Science News, El Pais (major Spanish Newspaper), MIT Technology Review

• 08/20/2015: Yuncheng Li et al. win the Best Paper Award at ICIMCS 2015 for the work on Using User Generated Photos to Monitor Air Pollution in Cities
• 07/20/2015: Deadline of the International Workshop on Social Multimedia Data Mining at ICDM 2015 (Atlantic City)

• 07/14/2015: Quanzeng You is invited to the 2015 Microsoft Research Asia Ph.D. Forum among "30 of the best Asian Ph.D. students from the Asia-Pacific region and the U.S."
• 05/15/2015: Prof. Luo is named a Data Science CoE Distinguished Researcher @Goergen Institute for Data Science
• 05/12/2015: Tianran Hu receives a student travel scholarship to attend ICWSM-2015
• 02/15/2015 Our work on computational image sentiment analysis is covered by Tech Times,  Science 2.0
• 02/03/2015: Communications of ACM coverage of our Non-Obtrusive Mental Health Monitoring work; also Gizmag

    2011-2014

• 12/15/2014: ICDM Tutorial on "Social Multimedia as Sensors"
• 12/14/2014: Workshop on Social Multimedia Data Mining at ICDM 2014
• 12/07/2014: Quanzeng You receives a student travel scholarship to attend AAAI-2015
• 10/02/2014: Prof. James Rehg visits our lab
•  08/28/2014: Guess how many students I currently advise?
•  07/15/2014: 2014 IEEE Multimedia Prize Paper Award

•  06/28/2014: The New Big Three of Rochester (Democratic and Chronicle interview)
•  05/01/2014: How Good is URCS? (a little known fact that puts us ahead of a perennial top 5 CS department)

•  12/20/2013: Prof. Roland Goecke visits our lab
•  10/25/2013: Prof. Bill Freeman visits our lab
•  10/18/2013: RoCData II (Rochester Big Data Forum)
•  10/17/2013: Prof. Ruslan Salakhutdinov visits our lab
•  10/14/2013: Prof. Honglak Lee  visits  our lab
•  08/29/2013: Prof. Eckehard Steinbach  visits  our lab
•  10/18/2013: RoCData II (Rochester Big Data Forum)
•  05/15/2013: The TMM Special Issue on "Social Media as Sensors" is complete and in press for August 2013

•  04/08/2013: A New Publication highlighting Big Data research at the UR
•  12/31/2012: URCS Faculty Recruiting in HCI and Big Data
•  10/04/2012: RoCData (Rochester Big Data Forum, speaker list)
•  09/19/2012: "Computer Vision with a Third Eye"

•  09/10/2012: VIStA (Visual Intelligence & Social Multimedia Analytics) Research Group is formed

• 02/25/2012: Professor Luo will co-chair the IEEE CVPR 2012 Area Chair meeting in Long Beach, CA

• 12/08/2011: Professor Luo will serve on the program committee for both the research track and industry track of the 2012 ACM SIGKDD conference
• 09/06/2011: I am looking for two self-motivated PhD students to join my research group in Fall 2012 to work in the areas of  computer vision and social media data mining.  Applicants are expected to have strong skills in both programming and mathematics. Related research publications in ranked venues of computer vision, machine learning, or data mining is a plus. To apply, check http://www.cs.rochester.edu/education/phd-program.

• 09/01/2011: Professor Luo joined the CS Department after a prolific career of 15+ years at Kodak Research Labs. 
• 09/01/2011: Professor Luo will co-chair the Third ACM SIGMM Workshop on Social Media (WSM2011) in conjunction with ACM Multimedia 2011
• 09/01/2011: A feature article on "Aesthetics and Emotions in Images: A Computational Perspective" is published in the IEEE Signal Processing Magazine
• 09/01/2011: A mini book on Interactive Co-segmentation of Objects in Image Collections (SpringerBriefs in Computer Science) by Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jiebo Luo, and Tsuhan Chen is available for pre-order
• 04/06/2011: A new book onSocial Media Modeling and Computingby Steven C.H. Hoi, Jiebo Luo, Susanne Boll and Dong Xu is published by Springer
   

Biography

Teaching

• Spring 2012 csc449 Machine Vision
• Spring 2012 csc249 Machine Vision
• Fall 2012 csc297 Intro. to Data Mining
• Fall 2012 csc577 Intro. to Data Mining
• Spring 2013 csc449 Machine Vision
• Spring 2013 csc249 Machine Vision
• Fall 2013 csc297 Intro. to Data Mining
• Fall 2013 csc577 Intro. to Data Mining
• Spring 2014 csc449 Machine Vision
• Spring 2014 csc249 Machine Vision
• Fall 2014 csc240 Data Mining
• Fall 2014 csc440 Data Mining
• Spring 2015 csc249/449 Machine Vision
• Fall 2015 csc240/440 Data Mining
• Spring 2016 csc249/449 Machine Vision
• Fall 2016 CSC240/440 Data Mining
• Fall 2016 DSC530 METHODS IN DATA-ENABLED RESEARCH INTO HUMAN BEHAVIOR AND ITS COGNITIVE AND NEURAL MECHANISMS
• Spring 2017 csc249/449 Machine Vision
• Fall 2017 CSC240/440 Data Mining
• Fall 2019 CSC440 Data Mining
• Spring 2021 CSC249/449 Machine Vision
• Spring 2022 CSC577 Advanced Topics in Computer Vision
• Fall 2022 CSC/DSC 440 Data Mining
• Spring 2023 CSC249/449 Machine Vision
• Fall 2023 CSC/DSC 440 Data Mining

Major Professional Services

• IEEE:
  
• IEEE Transactions on Multimedia (TMM), Editor-in-Chief (2020-2022), Editorial Board Member (2004-2008; 2013-2015)
• IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Editorial Board Member (2006-2011)
• IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), Editorial Board Member (2010-2012)
• IEEE Transactions on Big Data (TBD), Editorial Board Member (2018-2022)
  
• IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Program Co-Chair (2012), Area Chair (2008/2017/2019)
  
• IEEE International Conference on Image Processing (ICIP), Program Co-Chair (2017), Area Chair (2008-2012)
• IEEE International Conference on Acoustics, Speech & Signal Processing (ICASSP), Area Chair (2009-2012)
  
• IEEE International Conference on Multimedia and Expo (ICME), Special Sessions Chair (2006), General Co-Chair (2024)
  
• IEEE Signal Processing Society Technical Committees: MLSP (2005-2010), MMSP (2006-2012), IVMSP (2007-2013)
• IEEE International Conference on Data Mining (ICDM), Program Committee (2013-2022), Senior PC member (2015)
• ACM
  
• ACM Multimedia Conference, General Co-Chair (2018), Program Co-Chair (2010), Awards Chair (2024), Workshops Co-Chair (2012), Tutorial Co-Chair (2011)
• ACM Transactions on Intelligent Systems and Technology, Editorial Board Member (2015-2024)
• ACM/IAPR International Conference on Content-based Image and Video Retrieval (CIVR), General Chair (2008)
• ACM International Conference on Multimedia Retrieval (ICMR), Program Co-Chair (2016), General Co-Chair (2023)
  
• ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), Senior PC member (2018-2020)
• ACM Web Conference (formerly WWW), PC member (2020- )
  
• ACM SIGMM Workshop on Social Media (WSM), Founding Chair (2009-2011)
  
• ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), Program Committee (2012- )
  
• ACM Conference on Information and Knowledge Management (CIKM), Program Committee (2013- )
  
• ACM International Conference on Multimedia Retrieval (ICMR), Steering Committee (2009- )
  
• AAAI
  
• International Conference on Artificial Intelligence (AAAI), Program Committee  (2013-2015), Senior PC / AC (2016- )
  
• International Joint Conference on Artificial Intelligence (IJCAI), Program Committee  (2013-), Senior PC (2015/2018)
• International Conference on Web and Social Media (ICWSM), Program Committee (2018), Senior PC (2019- )
• SPIE
• SPIE International Symposium on Visual Communication and Image Processing (VCIP), Co-Chair (2007)
• SPIE International Symposium on Multimedia Systems & Applications, General Co-Chair (2007)
  
• Journal of Electronic Imaging (JEI), Editorial Board Member (2003-2013)
• CVF
  
  • CVF/IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Program Co-Chair (2012), Area Chair (2008/2017/2019)
    
  • International Conference on Computer Vision (ICCV), Area Chair (2011) 
  • European Conference on Computer Vision, Area Chair (2020)
• IAPR
  
• Pattern Recognition, Editorial Board Member (2002-2019)
• International Conference on Pattern Recognition (ICPR), Area Chair (2014, 2016, 2018)
• MICCAI
• International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Area Chair (2020)
• ACL
• Annual meeting of the Association for Computational Linguistics (ACL), Program Committee (2020- )
• Conference on Empirical Methods in Natural Language Processing (EMNLP), Program Committee (2020- )

• NSF:
• CISE-IIS-RI, panelist
• CISE-IIS-III, panelist
• SCH, panelist
• VEC, panelist
• CPS, panelist
• Industry:
• Board Member, Greater Rochester Data Science Industry Consortium (2018-2019)
  
• Advisor, Tencent (Youtu) (2017)
• Visiting Professor, Tencent AI Lab (2018)
• Visiting Professor, Facebook AI (2021-2022)
  

Books

• Deep Neural Network for Medical Image Computing: Principles and Applications (Elsevier, 2022)
  
• Social Media Modeling and Computing (Springer)
• Interactive Co-segmentation of Objects in Image Collections (SpringerBriefs in Computer Science)
• Multimedia Interaction and Intelligent User Interfaces (Springer Advances in Computer Vision and Pattern Recognition)
• Computer Vision [in English])[amazon.cn]

Selected Recent Publications (Publications listed more comprehensively by DBLP)

My Citations

surveys

• Aesthetics and Emotions in Images: A Computational Perspective
  

•  Geotagging in Multimedia and Computer Vision -- A Survey
  

• Small Data Challenges in Big Data Era: Recent Progress on Unsupervised and Semi-Supervised Methods
  

• Human Behavior in the Time of COVID-19: Learning from Big Data

• GPT-4V(ision) as A Social Media Analysis Engine

• Large Language Models in Medicine: Principles, Applications, and Challenges

• Video Understanding with Large Language Models: A Survey
  
  

computer vision, NLP and media understanding

• Fenglin Liu, Zheng Li, Qingyu Yin, Jinfa Huang, Xian Wu, Anshul Thakur, Kim Branson, Patrick Schwab, Bing Yin, Yefeng Zheng, Jiebo Luo, and David Clifton, "A Multimodal Multidomain Multilingual Medical Foundation Model for Zero Shot Clinical Diagnosis", npj Digital Medicine, in press.
• Yongsheng Yu*, Ziyun Zeng*, Hang Hua, Jianlong Fu, Jiebo Luo, "PromptFix: You Prompt and We Fix the Photo," Conference on Neural Information Processing Systems (NeurIPS), Vancouver, Canada, December 2024.
• Shenghai Yuan, Jinfa Huang, Yongqi Xu, YaoYang Liu, Shaofeng Zhang, Yujun Shi, Ruijie Zhu, Xinhua Cheng, Jiebo Luo, Li Yuan, "ChronoMagic: A Benchmark for Metamorphic Evaluation of Time-lapse Text-to-Video Generation," Dataset & Benchmark Track, Conference on Neural Information Processing Systems (NeurIPS), Vancouver, Canada, December 2024. (Spotlight)
  
• Daoan Zhang*, Junming Yang*, Hanjia Lyu*, Zijian Jin, Yuan Yao, Mingkai Chen, and Jiebo Luo, "A Benchmark and Chain-of-Thought Prompting Strategy for Large Multimodal Models with Multiple Image Inputs," International Conference on Pattern Recognition (ICPR), Kolkata, India, December 2024.
• Jiayi Hao, Yulin Feng, Wenbin Li, Jiebo Luo, "Rare Fungi Image Classification Based on Few-Shot Learning and Data Augmentation," International Conference on Pattern Recognition (ICPR), Kolkata, India, December 2024.
• Jie An, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Lijuan Wang, Jiebo Luo, "OpenLEAF: A Novel Benchmark for Open-Domain Interleaved Image-Text Generation," Brave New Ideas Track, ACM Multimedia Conference, Melbourne, Australia, October 2024.
  
• Zixuan Wang, Jiayi Li, Xiaoyu Qin, Shikun Sun, Songtao Zhou, Jia Jia, Jiebo Luo, "DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis," ACM Multimedia Conference, Melbourne, Australia, October 2024.
  
• Hang Hua, Jing Shi, Kushal Kafle, Simon Jenni, Daoan Zhang, John Collomosse, Scott Cohen, Jiebo Luo， "FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction," European Conference on Computer Vision (ECCV), Milan, Italy, September 2024.
  
• Xinnong Zhang*, Haoyu Kuang*, Xinyi Mou, Hanjia Lyu, Kun Wu, Siming Chen, Jiebo Luo, Xuanjing Huang, and Zhongyu Wei, "SoMeLVLM: A Large Vision Language Model for Social Media Processing," Annual Meeting of the Association for Computational Linguistics (ACL), Bangkok, Thailand, August 2024.
• Wenting Chen, Linlin Shen, Jingyang Lin, Jiebo Luo, Xiang Li, Yixuan Yuan, "Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generation," Annual Meeting of the Association for Computational Linguistics (ACL), Bangkok, Thailand, August 2024.
• Junyu Chen, Jie An, Hanjia Lyu, Christopher Kanan, Jiebo Luo, "Holistic Visual-Textual Sentiment Analysis with Prior Models," IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR), San Jose, CA, August 2024.
  
• Yiwei Han, Kaiyi Qi, Jiebo Luo, "Plastic Surgery Image Classification and Generation," IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR), San Jose, CA, August 2024.
  
• Jie An, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Zicheng Liu, Lijuan Wang, Jiebo Luo, "Bring Metric Functions into Diffusion Models," 33rd International Joint Conference on Artificial Intelligence (IJCAI), Jeju, South Korea, August 2024.
  
• Yuying Li, Zeyan Liu, Junyi Zhao, Liangqin Ren, Fengjun Li, Jiebo Luo, and Bo Luo, "The Adversarial AI-Art: Understanding, Generation, Detection, and Benchmarking," 29th European Symposium on Research in Computer Security (ESORICS), Bydgoszcz, Poland, September 2024.
  
• Junyu Chen, Jie An, Hanjia Lyu, and Jiebo Luo, "Learning to Evaluate the Artness of AI-generated Images," IEEE Transactions on Multimedia, Early Access, 2024.
• Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Eli Shechtman, Connelly Barnes, Jianming Zhang, Qing Liu, Yuqian Zhou, Sohrab Amirghodsi, Jiebo Luo, "Structure-Guided Image Completion with Image-level and Object-level Semantic Discriminators," IEEE Transactions on Pattern Analysis and Machine Intelligence, Early Access, 2024.
• Shuo Yang, Zirui Zhang, Xinxiao Wu, Jiebo Luo, "Dynamic Pathway for Query-aware Feature Learning in Language-driven Action Localization," IEEE Transactions on Multimedia, Early Access, 2024.
• Ali Vosoughi, Shijian Deng, Songyang Zhang, Yapeng Tian, Chenliang Xu, Jiebo Luo, "Cross Modality Bias in Visual Question Answering: A Causal View with Possible Worlds VQA," IEEE Transactions on Multimedia, Early Access, 2024.
• Yuheng Shi, Xinxiao Wu. Jiebo Luo, "Knowledge Prompting for Few-shot Action Recognition in Videos," IEEE Transactions on Multimedia, Early Access, 2024.
• Yujian Feng, Feng Chen, Yimu Ji, Fei Wu, Tianliang Liu, Shangdong Liu, Xiao-Yuan Jing, Jiebo Luo, "Cross-modality Spatial-temporal Transformer for Video-based Visible-infrared Person Re-identification, " IEEE Transactions on Multimedia, Early Access, 2024.
• Peggy Tang, Kun Hu, Lei Zhang, Jiebo Luo, Zhiyong Wang, "TLDW: Extreme Multimodal Summarisation of News Videos," IEEE Transactions on Circuits and Systems for Video Technology, 34(3), March 2024.
  
• Hanjia Lyu, Song Jiang, Hanqing Zeng, Yinglong Xia, Qifan Wang, Si Zhang, Ren Chen, Chris Leung, Jiajie Tang, and Jiebo Luo, "LLM-Rec: Personalized Recommendation via Prompting Large Language Models," Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, USA, June 2024.
• Yuan Yao, Yuanhan Zhang, Zhenfei Yin, Jiebo Luo, Wanli Ouyang, "3D Point Cloud Pre-training with Knowledge Distillation from 2D Images," IEEE International Conference on Multimedia and Expo (ICME), Niagara Falls, Canada, July 2024.
• Yongsheng Yu, Jiebo Luo, "Chain-of-Thought Prompting for Demographic Inference with Large Multimodal Models," IEEE International Conference on Multimedia and Expo (ICME), Niagara Falls, Canada, July 2024.
  
• Zixuan Wang, Jia Jia, Shikun Sun, Haozhe Wu, Rong Han, Zhenyu Li, Di Tang, Zhou Jiaqing, Jiebo Luo, "DanceCamera3D: 3D Camera Movement Synthesis with Music and Dance," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2024.
• Zhikai Chen, Fuchen Long, Zhaofan Qiu, Ting Yao, Wengang Zhou, Jiebo Luo, Tao Mei, "Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2024.
  
• Xinyi Mou, Zejun Li, Hanjia Lyu, Jiebo Luo, Zhongyu Wei, "Unifying Local and Global Knowledge: Empowering Large Language Models as Political Experts with Knowledge Graphs," The Web Conference (WWW), Singapore, May 2024.
• Hanqing Zeng*, Hanjia Lyu*, Diyi Hu, Yinglong Xia, and Jiebo Luo, "Mixture of Weak & Strong Experts on Graphs," Twelfth International Conference on Learning Representations (ICLR), Vienna, Austria, May 2024.
• Shaofeng Zhang, Jinfa Huang, Qiang Zhou, Zhibin Wang, Fan Wang, Jiebo Luo, Junchi Yan, "Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach," Twelfth International Conference on Learning Representations (ICLR), Vienna, Austria, May 2024.
  
• Wenxi Yue, Jing Zhang, Kun Hu, Yong Xia, Jiebo Luo, Zhiyong Wang, "SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation," The 38th AAAI Conference on Artificial Intelligence (AAAI), February 2024.
• Cong Jin, Ruolin Zhu, Zixing Zhu, Min Yang, Minghui Qiu, Jiebo Luo, "MtArtGPT: A Multi-task Art Generation System with Pre-Trained Transformer," IEEE Transactions on Circuits and Systems for Video Technology, Early Access, 2024.
  
• Zhikai Chen, Fuchen Long, Zhaofan Qiu, Juyong Zhang, Zheng-jun Zha, Ting Yao, Jiebo Luo, "A Closer Look at the Reflection Formulation in Single Image Reflection Removal," IEEE Transactions on Image Processing, Early Access, 2024.

    2023

• Zhikai Chen, Fuchen Long, Zhaofan Qiu, Juyong Zhang, Zheng-jun Zha, Ting Yao, Jiebo Luo, "A Closer Look at the Reflection Formulation in Single Image Reflection Removal," IEEE Transactions on Image Processing, Early Access, 2023.
  
• Jingyang Lin, Hang Hua, Ming Chen, Yikang Li, Mark Hsiao, Chiuman He, Jiebo Luo, "VideoXum: Cross-modal Visual and Textural Summarization of Videos," IEEE Transactions on Multimedia, Early Access. [Project Page]
  
• Hang Hua, Xingjian Li, Dejing Dou, Cheng-Zhong Xu, Jiebo Luo, "Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization," IEEE Transactions on Neural Networks and Learning Systems, Early Access.
• Mohammad Mahdi Kamani, Yuhang Yao, Hanjia Lyu, Zhongwei Cheng, Lin Chen, Liangju Li, Carlee Joe-Wong, Jiebo Luo, "Wyze Rule: Federated Rule Dataset for Rule Recommendation Benchmarking," Neural Information Processing Systems (NeurlPS), December 2023.
  
• Alexander Martin, Haitian Zheng, Jie An, Jiebo Luo, "Jurassic World Remake: Bringing Ancient Fossils Back to Life via Zero-Shot Long Image-to-Image Translation," ACM Multimedia Conference, Brave New Ideas track, Ottawa, Canada, October 2023. (Oral Presentation)
• Wenbin Li, Jing Huo, Yang Gao, Lei Wang, Jiebo Luo, "LibFewShot: A Comprehensive Library for Few-shot Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, Early Access.
  
• Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah Smith, Jiebo Luo, "Prompt-Guided Image Captioning for VQA with GPT-3," International Conference on Computer Vision (ICCV), Paris, France, October 2023.
  
• Pingyu Wu, Wei Zhai, Yang Cao, Jiebo Luo, Zheng-Jun Zha, "Spatial-Aware Token for Weakly Supervised Object Localization," International Conference on Computer Vision (ICCV), Paris, France, October 2023.
  
• Yuhang Yang, Wei Zhai, Hongchen Luo, Yang Cao, Jiebo Luo, Zheng-Jun Zha, "Grounding 3D Object Affordance from 2D Interactions in Images," International Conference on Computer Vision (ICCV), Paris, France, October 2023.
  
• Peggy Tang, Kun Hu, Lei Zhang, Jiebo Luo, Zhiyong Wang, "TL;DW: Extreme Multimodal Summarisation of News Videos," IEEE Transactions on Circuits and Systems for Video Technology, in press.
  
• Siyu Huang, Jie An, Donglai Wei, Jiebo Luo, Hanspeter Pfister, "QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, June 2023.
• Zhikai Chen, Fuchen Long, Zhaofan Qiu, Ting Yao, Wengang Zhou, Jiebo Luo, Tao Mei, "AnchorFormer: Point Cloud Completion from Discriminative Nodes," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, June 2023.
• Hongwei Xue, Peng Gao, Hongyang Li, Yu Qiao, Hao Sun, Houqiang Li, Jiebo Luo, "Stare at What You See: Masked Image Modeling without Reconstruction," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, June 2023.
• Chengzhi Cao, Xueyang Fu, Hongjian Liu, Yukun Huang, Kunyu Wang, Jiebo Luo, Zheng-Jun Zha, "Event-guided Person Re-Identification via Sparse-Dense Complementary Learning," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, June 2023.
• Jin Chen, Zhi Gao, Xinxiao Wu, Jiebo Luo, "Meta-causal Learning for Single Domain Generalization," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, June 2023.
  
• Hongwei Xue, Yuchong Sun, Bei Liu, Jianlong Fu, Ruihua Song, Houqiang Li, Jiebo Luo, "CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Alignment," The Eleventh International Conference on Learning Representations (ICLR), Kigali, Rwanda, May 2023.
  
• Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Jianming Zhang, Ning Xu, Jiebo Luo, "Semantic Layout Manipulation with High-resolution Sparse Attention," IEEE Transactions on Pattern Analysis and Machine Intelligence, March 2023.
  
• Hao Wang, Zheng-Jun Zha, Liang Li, Xuejin Chen, Jiebo Luo, "Semantic and Relation Modulation for Audio-Visual Event Localization," IEEE Transactions on Pattern Analysis and Machine Intelligence, February 2023.
  
• Jie An, Tao Li, Haozhi Huang, Jinwen Ma, and Jiebo Luo, "Is Bigger Always Better? An Empirical Study on Efficient Architectures for Style Transfer and Beyond," IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, Hawaii, January 2023. 
  

• Songyang Zhang, Linfeng Song, Lifeng Jin, Haitao Mi, Kun Xu, Dong Yu and Jiebo Luo, "Learning a Grammar Inducer from Massive Uncurated Instructional Videos," The Conference on Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, December 2022. 
  
• Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Eli Shechtman, Connelly Barnes, Jianming Zhang, Ning Xu, Sohrab Amirghodsi, Jiebo Luo, "Image Inpainting with Cascaded Modulation GAN and Object-Aware Training," European Conference on Computer Vision (ECCV), Tel-Aviv, October 2022. 
  
• Hanjia Lyu, Jiebo Luo, "Understanding Political Polarization via Jointly Modeling Users, Connections and Multimodal Contents on Heterogeneous Graphs," ACM International Conference on Multimedia (ACM MM), Lisbon, Portugal, October 2022.
• Zhaoyi Wan, Dejia Xu, Jian Wang, and Jiebo Luo. "Cloud2Sketch: Augmenting Clouds with Imaginary Sketches," ACM International Conference on Multimedia (ACM MM), Lisbon, Portugal, October 2022.
  
• Wei Zhu, Zihe Zheng, Haitian Zheng, Hanjia Lyu, Jiebo Luo, "Learning to Aggregate and Refine Noisy Labels for Visual Sentiment Analysis," International Conference on Pattern Recognition (ICPR), Montreal, Quebec, Canada, August 2022.
  
• Zhengyuan Yang, Jingen Liu, Jing Huang, Xiaodong He, Tao Mei, Chenliang Xu, Jiebo Luo, "Cross-modal Contrastive Distillation for Instructional Activity Anticipation," International Conference on Pattern Recognition (ICPR), Montreal, Quebec, Canada, August 2022.
  
• Wei Xiong, Ding Liu, Xiaohui Shen, Chen Fang, Jiebo Luo, "Unsupervised Low-light Image Enhancement with Decoupled Networks," International Conference on Pattern Recognition (ICPR), Montreal, Quebec, Canada, August 2022.
  
• Zhongjie Yu, Gaoang Wang, Lin Chen, Jiebo Luo, Sebastian Raschka, "When Few-Shot Learning Meets Video Object Detection," International Conference on Pattern Recognition (ICPR), Montreal, Quebec, Canada, August 2022.
  
• Wei Zhu, Le Lu, Jing Xiao, Mei Han, Jiebo Luo, Adam Harrison, "Localized Adversarial Domain Generalization," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
• Kai Zhu, Yang Cao, Wei Zhai, Jiebo Luo, Zheng-jun Zha, "Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
• Fuchen Long, Zhaofu Qiu, Yingwei Pan, Ting Yao, Jiebo Luo, Tao Mei. "Stand-Alone Inter-Frame Attention in Video Models," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
• Jing Shi, Ning Xu, Haitian Zheng, Alex Smith, Jiebo Luo, Chenliang Xu, "SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
• Shaofei Cai, Liang Li, Xinzhen Han, Jiebo Luo, Zheng-Jun Zha, Qingming Huang, "Automatic Relation-aware Graph Network Proliferation," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
  

• Sheng Liu, Zhongwei Cheng, Lin Chen, Jiebo Luo, Junsong Yuan, "A Compositional Model for Visual Relationship Grounding," IEEE Transactions on Image Processing, in press.
• Songyang Zhang, Houwen Peng, Yijuan Lu, Jianlong Fu, Jiebo Luo, "Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
• Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo, "Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training," Neural Information Processing Systems (NeurlPS), December 2021.
• Wentian Zhao, Xinxiao Wu, Jiebo Luo, "Multi-modal Dependency Tree for Video Captioning," Neural Information Processing Systems (NeurlPS), December 2021.
  
• Jian Ding, Nan Xue, Gui-Song Xia, Xiang Bai, Wen Yang, Michael Yang, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, Liangpei Zhang, "Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
  
• Litao Yu, Zhibin Li, Min Xu, Yongsheng Gao, Jiebo Luo, Jian Zhang, "Distribution-aware Margin Calibration for Semantic Segmentation in Images," International Journal of Computer Vision, in press.
  
• Zhengyuan Yang, Songyang Zhang, Liwei Wang, Jiebo Luo, "SAT: 2D Semantics Assisted Training for 3D Visual Grounding," International Conference on Computer Vision (ICCV), 2021. (oral presentation)
• Wei Zhu, Haitian Zheng, Haofu Liao, Weijian Li, Jiebo Luo, "Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization," International Conference on Computer Vision (ICCV), 2021.
• Heliang Zheng, Huan Yang, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo, "Learning Dual-Path Embedding Network for Degraded-Reference Image Quality Assessment," International Conference on Computer Vision (ICCV), 2021.
• Jing Bi, Jiebo Luo, Chenliang Xu, "Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learning," International Conference on Computer Vision (ICCV), 2021. (oral presentation)
  
• Mengshi Qi, Jie Qin, Di Huang, Zhiqiang Shen, Yi Yang, and Jiebo Luo, "Latent Memory-augmented Graph Transformer for Visual Storytelling," ACM Multimedia Conference, Chengdu, China, October 2021.
• Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo, "Learning Fine-Grained Motion Embedding for Landscape Animation," ACM Multimedia Conference, Chengdu, China, October 2021. (oral presentation)
• Songyang Zhang, Linfeng Song, Lifeng Jin, Dong Yu, Jiebo Luo, "Video-aided Unsupervised Grammar Induction," Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, June 2021. (Best Long Paper)
  
• Hang Hua, Xingjian Li, Dejin Dou, Chengzhong Xu, Jiebo Luo, "Noise Stability Regularization for Improving BERT Fine-tuning," Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, June 2021.
  
• Zhengyuan Yang, Yijuan Lu, Xi Yin, Dinei Florencio, Lijuan Wang, Cha Zhang, Lei Zhang, Jiebo Luo, "TAP: Text-Aware Pre-training for Text-VQA and Text-Caption," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2021. (oral presentation)
• Jie An, Siyu Huang, Yibing Song, Dejing Dou, Wei Liu, Jiebo Luo, "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2021.
• Jing Wang, Jinhui Tang, Mingkun Yang, Xiang Bai, Jiebo Luo, "Improving OCR-based Image Captioning by Incorporating Geometrical Relationship," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2021.
• Hao Wang, Zheng-Jun Zha, Liang Li, Dong Liu, Jiebo Luo, "Structured Multi-Level Interaction Network for Video Moment Localization via Language Query," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2021.
  
• Kecheng Zheng, Wu Liu, Lingxiao He, Tao Mei, Jiebo Luo, Zheng-Jun Zha, "Group-aware Label Transfer for Domain Adaptive Person Re-identification, " IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2021.
• Cheng Peng, Haofu Liao, Gina Wong, Jiebo Luo, S. Kevin Zhou, Rama Chellappa, "XraySyn: Realistic View Synthesis from a Single Radiograph through CT Prior," The 35th AAAI Conference on Artificial Intelligence (AAAI), February 2021.
• Jin Chen, Xinxiao Wu, Yao Hu, Jiebo Luo, ''Spatial-temporal Causal Inference for Partial Image-to-video Adaptation," The 35th AAAI Conference on Artificial Intelligence (AAAI), February 2021.
• Haitian Zheng, Lele Chen, Chenliang Xu, Jiebo Luo, "Pose Flow Learning from Person Images for Posed Guided Synthesis," IEEE Transactions on Image Processing, 30:1898-1909, 2021.
  
• Yanlong Dong, Ying Zhang, Lin Ma, Zhi Wang, Jiebo Luo, ''Unsupervised text-to-image synthesis,'' Pattern Recognition, 110: 107573, 2021.

• Zhengyuan Yang, Tushar Kumar, Tinalang Chen, Jinsong Su, Jiebo Luo, ''Grounding-Tracking-Integration,'' IEEE Transactions on Circuits and Systems for Video Technology, in press.
  
• Xinxiao Wu, Ruiqi Wang, Jingyi Hou, Hanxi Lin, Jiebo Luo, ''Spatial-Temporal Relation Reasoning for Action Prediction in Videos,'' International Journal of Computer Vision (IJCV), in press.
  
• Jie An, Tianlang Chen, Songyang Zhang, Jiebo Luo, "Global Image Sentiment Transfer," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
  
• Mingkun Yang, Haitian Zheng, Xiang Bai, Jiebo Luo, "Cost-Effective Adversarial Attacks against Scene Text Recognition," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
  
• Gaoang Wang, Lin Chen, Mingwei He, Jiebo Luo, "DAIL: Dataset-Aware and Invariant Learning for Face Recognition," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
  
• Weijian Li, Haofu Liao, Shun Miao, Le Lu, Jiebo Luo, "Unsupervised Learning of Landmarks based on Inter-Intra Subject Consistencies," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
  
• Zhengyuan Yang, Yuncheng Li, Linjie Yang, Ning Zhang, Jiebo Luo, "Weakly Supervised Body Part Segmentation with Pose based Part Priors," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
• Zhengyuan Yang, Amanda Kay, Yuncheng Li, Wendi Cross, Jiebo Luo, "Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
• Heliang Zheng, Jianlong Fu, Yanhong Zeng, Zheng-jun Zha, Jiebo Luo, "Learning Semantic-aware Normalization for Generative Adversarial Networks," Neural Information Processing Systems (NeurlPS), December 2020. (spotlight presentation)
• Yiming Xu, Lin Chen, Zhongwei Cheng, Lixin Duan, Jiebo Luo, "Open-Ended Visual Question Answering by Multi-Modal Domain Adaptation," The Conference on Empirical Methods in Natural Language Processing (EMNLP), Findings of EMNLP, November 2020.
  
• Xiankai Lu, Wenguan Wang, Jianbing Shen, David Crandall, Jiebo Luo, "Zero-Shot Video Object Segmentation with Co-Attention Siamese Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
• Wenbin Li*, Wei Xiong*, Haofu Liao, Jing Huo, Yang Gao, Jiebo Luo, "CariGAN: Caricature Generation through Weakly Paired Adversarial Learning," Neural Networks, in press.
  
• Tianlang Chen, Wei Xiong, Haitian Zheng, Jiebo Luo, "Image Sentiment Transfer," Brave and New Ideas Track, ACM Multimedia Conference, Seattle, WA, October 2020. (Oral Presentation)
• Jing Wang, Jinhui Tang, Jiebo Luo, "Multimodal Attention with Image Text Spatial Relationship for OCR-Based Image Captioning," ACM Multimedia Conference, Seattle, WA, October 2020.
• Mengshi Qi, Jie Qin, Xiantong Zhen, Di Huang, Yi Yang, Jiebo Luo, "Few-Shot Ensemble Learning for Video Classification with SlowFast Memory Networks," ACM Multimedia Conference, Seattle, WA, October 2020.
  
• Huan Lin, Fandong Meng, Jinsong Su, Yongjing Yin, Zhengyuan Yang, Yubin Ge, Jie Zhou, Jiebo Luo, "Dynamic Context-guided Capsule Network for Multimodal Machine Translation," ACM Multimedia Conference, Seattle, WA, October 2020.
• Rui Zhao, Kecheng Zheng, Zheng-Jun Zha, Hongtao Xie, Jiebo Luo, "Memory Enhanced Embedding Learning for Cross-Modal Video-Text Retrieval," ACM Multimedia Conference, Seattle, WA, October 2020.
• Yangchun Zhu, Zheng-Jun Zha, Tianzhu Zhang, Jiawei Liu, Jiebo Luo, "A Structured Graph Attention Network for Vehicle Re-Identification," ACM Multimedia Conference, Seattle, WA, October 2020. (Oral Presentation)
• Hao Wang, Zheng-Jun Zha, Xuejin Chen, Zhiwei Xiong, Jiebo Luo, "Dual Path Interaction Network for Video Moment Localization," ACM Multimedia Conference, Seattle, WA, October 2020.
  
• Haitian Zheng, Lele Chen, Chenliang Xu, Jiebo Luo, "Pose Flow Learning from Person Images for Posed Guided Synthesis," IEEE Transactions on Image Processing, in press.
  
• Fuchen Long, Ting Yao, Zhaofan Qiu, Xinmei Tian, Jiebo Luo, Tao Mei, "Learning to Localize Actions from Moments," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020. (oral presentation)
• Jianxin Lin, Yingxue Pang, Yingce Xia, Zhibo Chen, Jiebo Luo, "TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020. (spotlight presentation)
• Zhengyuan Yang, Tianlang Chen, Liwei Wang, Jiebo Luo, "Improving One-stage Visual Grounding by Recursive Sub-query Construction," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020.
• Tianlang Chen, Jiajun Deng, Jiebo Luo, "Adaptive Offline Quintuplet Loss for Image-text Matching," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020.
• Haitian Zheng, Haofu Liao, Lele Chen, Wei Xiong, Tianlang Chen, Jiebo Luo, "Example-Guided Image Synthesis across Arbitrary Scenes using Masked Spatial-Channel Attention and Self-Supervision," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020.
• Weijian Li, Yuhang Lu, Kang Zheng, Haofu Liao, Chihung Lin, Jiebo Luo, Chi-Tung Cheng, Jing Xiao, Le Lu, Chang-Fu Kuo, Shun Miao, "Cross-domain Structured Landmark Detection via Progressive Topology-Adapting Deep Graph Learning," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020.
• Mang Ye, Jianbing Shen, David J. Crandall, Ling Shao, Jiebo Luo, "Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020.
• Huafeng Li, Jiajia Xu, Zhengtao Yu, and Jiebo Luo, "Jointly Learning Commonality and Specificity Dictionaries for Person Re-Identification," IEEE Transactions on Image Processing, in IEEE Xplore.
  
• Xin Liu, Kai Liu, Jinsong Su, Yebin Ge, Xiang Li, Bin Wang, Jiebo Luo, "An Iterative Multi-Source Mutual Knowledge Transfer Framework for Machine Reading Comprehension," International Joint Conference on Artificial Intelligence (IJCAI), Yokohama, Japan, July 2020.
  
• Yongjing Yin, Fandong Meng, Jinsong Su, Chunlun Zhou, Zhengyuan Yang, Jie Zhou, Jiebo Luo, "A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation," Annual Meeting of the Association for Computational Linguistics (ACL), Seattle, WA, July 2020.
• Mengshi Qi, Yunhong Wang, Annan Li, Jiebo Luo, "STC-GAN: Spatio-temporally coupled Generative Adversarial Networks for Predictive Scene Parsing," IEEE Transactions on Image Processing, 29: 5420-5430, 2020.
• Jingyi Hou, Xinxiao Wu, Ruiqi Wang, Jiebo Luo, Yunde Jia, "Confidence-Guided Self Refinement for Action Prediction in Untrimmed Videos," IEEE Transactions on Image Processing, in press.
  
• Wei Xiong, Yutong He, Yixuan Zhang, Wenhan Luo, Lin Ma, Jiebo Luo, "Fine-grained Image-to-Image Transformation towards Visual Recognition," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020.
• Jie Chen, Zhiheng Li, Chenliang Xu, Jiebo Luo, "Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020. (oral presentation)
• Zhongjie Yu, Lin Chen, Zhongwei Cheng, Jiebo Luo, "TransMatch: A Transfer-Learning Scheme for Semi-Supervised Few-Shot Learning," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020.
• Zhaoyi Wang, Jielei Zhang, Liang Zhang, Cong Yao, Jiebo Luo, "On Vocabulary Reliance in Scene Text Recognition," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020.
• Jiamin Wu, Tianzhu Zhang, Zheng-Jun Zha, Jiebo Luo, Yongdong Zhang, Feng Wu, "Self-supervised Domain-aware Generative Network for Generalized Zero-shot Learning," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020.
  
• Jie An, Haoyi Xiong, Jun Huan, Jiebo Luo, "Ultrafast Photorealistic Style Transfer via Neural Architecture Search," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020. (oral presentation)
• Tianlang Chen, Jiebo Luo, "Expressing Objects just like Words: Recurrent Visual Embedding for Image-Text Matching," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020.
• Songyang Zhang, Houwen Peng, Jianlong Fu, Jiebo Luo, "Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020.
• Jinyi Hou, Xinxiao Wu, Yunde Jia, Jiebo Luo, "Joint Commonsense and Relation Reasoning for Image and Video Captioning," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020.
• Jiali Zeng, Linfeng Song, Jinsong Su, Xie Jun, Wei Song, Jiebo Luo, "Neural Simile Recognition with Cyclic Multitask Learning and Local Attention," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020.
• Yongjing Yin, Fandong Meng, Jinsong Su, Yubin Ge, Linfeng Song, Jie Zhou, Jiebo Luo, "Enhancing Pointer Network for Sentence Ordering with Pairwise Ordering Predictions," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020.
  
• Heliang Zheng, Zheng-Jun Zha, Jiebo Luo, "Learning Rich Part Hierarchies with Progressive Attention Networks for Fine-Grained Image Recognition," IEEE Transactions on Image Processing, 29: 476-488, 2020.
• Tianliang Liu, Quanzeng You, Yanzhang Wang, Xiaodong Dong, Xiubin Dai, Jiebo Luo, "Sentiment Recognition for Short Annotated GIFs Using Visual-Textual Fusion," IEEE Transactions on Multimedia, 22(4): 1098-1110, 2020.