Facebook Twitter Instagram
    Trending
    • Innokin Launches “Smoke-free 2023” Campaign in Support of Stoptober
    • Sabrent Rocket V90 256GB SD UHS-II Memory Card and SD/MicroSD UHS-II Card Reader Review
    • Glamee Dice 6000 vs. Glamee Mini Disposable Vapes Comparison Review
    • Smoant Pasito Mini vs. Smoant Charon Baby Plus Vape: Comparison Review
    • Get the WLtoys A959-B 4WD RC Car at €53.93 From TOMTOP in Flash Sale
    • Get Ready to Ride with the HAPPYRUN TANK G60 Electric Bike at €1089.99
    • Grab the Bisofice K6 Laser Engraver & Logo Marker Machine at €99.00 with FREE SHIPPING
    • Get Lenovo ThinkBook 16+ Laptop at €719 with Free Shipping From TOMTOP
    Facebook YouTube
    Login Register
    IGeeKphone China Phone, Tablet PC, VR, RC Drone News, Reviews
    • HOME
    • NEWS
      • ChatGPT
      • Minecraft
    • PHONE
      • Top Phones For Your First Choice
      • Phone Comparison
      • Xiaomi
      • Doogee
      • Blackview
      • Black Shark
      • FOSSIBOT
      • Tomtop
      • Cafago
      • Geekbuying
      • Banggood
      • RCMOMENT
      • Amazon
      • TEMU
      • TikTok
      • Aliexpress
      • Walmart
      • Newegg
      • MercadoLibre
      • Lazada
    • BEST VAPE
      • Vape News
      • Vape Market Trend
      • Expo News
      • Vape Deals
      • Vape Comparsion
      • Vape Guide
        • Guide For Beginners
        • Guide for Best Users
      • Giveaway
      • Best Vape Stores
      • Best Starter Vape Kits
      • Best Vapes for Beginners
      • Best Disposable Vapes
      • Best Pod Systems
      • Best Pod Mod Vapes
      • Best Vapes for Nic Salt E-Juice
      • Best Vapes to Quit Smoking
      • RDA vs. RDTA vs. RTA
    • Vape Brand
      • VOOPOO
      • MOTI
      • OXVA
      • ZOVOO
      • VAPESOURCING
        • Update Daily: Enjoy Coupon Code for Latest Vapes from Vapesourcing
      • VAPORESSO
      • SMOANT
      • BECO
      • Bangmavape
      • FREETON
      • INNOKIN
      • PUFFMI
      • ORIONBARTECH
      • MASKKING VAPE
      • GLAMEEVAPE
      • MRFOG
      • ANYX
      • YOCAN
    • REVIEW
      • E-cigar Review
      • Phones
      • Tablet PC
      • TV Box
      • RC Drone
      • Wearables
      • Camera
      • Accessories
      • VR Headset
    • MORE
      • E-CIGAR
      • GAMING
        • Top Gaming Products
      • 3D PRINTER
        • Anycubic
        • ATOMSTACK
        • Creality
        • Top 3D printer to Choose First
      • WEARABLES
        • OneOdio
        • BlitzWolf
        • Top Smartwatch for First Choice
      • TV BOX
        • Chuwi mini pc
        • Beelink
        • GMKTEC
        • MOREFINE
      • RC DRONE
        • MEPSKING
        • DJI
        • MJX
        • JJRC
        • Hubsan
        • Top RC Drone
      • CAMERA
        • Gopro
        • Insta360
        • Andoer
      • ACCESSORIES
      • VR HEADSET
      • ROM
        • SAMSUNG
        • XIAOMI
        • ASUS
        • MEIZU
        • LENOVO
        • HUAWEI
        • ONEPLUS
        • ZTE
        • UMIDIGI
        • DOOGEE
        • HOMTOM
        • ELEPHONE
        • ULEFONE
        • BLACKVIEW
        • VERNEE
        • LEAGOO
        • CHUWI
        • TECLAST
        • PIPO
        • TV BOX ROM
    • DEAL
    • 3D PRINTER
    • SMART HOME
      • ALLJOY
      • Govee
      • MiJia
      • Tronsmart
    • TABLET
      • Chuwi
      • INNOCN
      • Teclast
      • Top Tablet for Your First Choice
      • Tablet/Laptop Comparsion
    • GAMING
    • E-BIKE
      • Samebike
      • Happyrun
      • ENGWE
    • Guide
    • Shop
    IGeeKphone China Phone, Tablet PC, VR, RC Drone News, Reviews
    You are at:Home»ChatGPT»Visual ChatGPT: It Absorb All the AI Drawing Skills
    ChatGPT

    Visual ChatGPT: It Absorb All the AI Drawing Skills

    Farrukh AhmadBy Farrukh AhmadMarch 11, 2023
    Facebook Twitter Pinterest LinkedIn Tumblr Email

    ChatGPT Visual version connects ChatGPT and a series of Visual Foundation Models to enable sending and receiving images during chatting. For example,  Ask it: Can you generate a cat slice for me? As we can see, Immediately there are all the text and pictures.

    Besides, it can also adjust the picture according to the new text command: replace the cat with a dog.

    At the same time, I can understand pictures and have the ability to understand. For example, send a picture to it, and then ask what color is the motorcycle? It answers black.

    As mentioned above, it is the visual version of ChatGPT proposed by senior researchers of MSRA. By combining multiple visual models for ChatGPT and utilizing a prompt manager (Prompt Manager), they successfully enabled ChatGPT to handle various visual tasks.

    This work became popular as soon as it was released, and the number of stars on GitHub has exceeded 1.5k. A brief summary is the feeling of merging GPT and Dall-E~

    Knowing words and drawing pictures… Someone said:

    Isn’t this the ultimate meme maker?

    The trick is that the prompt works?

    Visual ChatGPT, in fact, enables ChatGPT to handle multimodal information. But training a multimodal model from scratch is a lot of work. The researchers thought that some visual models could be combined on the basis of ChatGPT.

    To achieve this goal, the key needs an intermediate station. As a result, they proposed the concept of a Prompt Manager.

    It has three main functions:

    First, clearly tell ChatGPT the role of each visual model, and specify the input and output formats.

    Second, convert different visual information, such as converting PNG images, depth images, mask matrices, etc. into language formats, which is convenient for ChatGPT to understand.

    Third, process the historical generation results of the visual model, as well as the call priority of different models, avoid conflicts, etc., so that ChatGPT can receive the generated content of the visual model in an iterative manner until the output is satisfactory to the user.

    In this way, the workflow of Visual ChatGPT looks like this:

    If the user enters a picture, the model will first send the content to the prompt manager, and then convert it into language for ChatGPT to judge. When it finds that the question does not need to call the visual model, it will directly give the output (the first answer).

    For the second question, ChatGPT needs to use a visual model to analyze the content of the question, so the visual model will start to execute, and then iterate until ChatGPT judges that it no longer needs to call the visual model, and then the result will be output.

    According to the paper, Visual ChatGPT contains 22 different visual models. Including Stable Diffusion, BLIP, pix2pix, etc. In order to verify the ability of Visual ChatGPT, they also conducted a large number of zero-shot experiments (zero-shot experiments).

    Results As shown at the beginning, Visual ChatGPT has a strong ability to understand images. The pictures can be continuously generated and modified according to the needs of people.

    Of course, the researchers also mentioned that there are still some limitations in this work. For example, the quality of generated results mainly depends on the performance of the visual model. And the use of a large number of hint projects will affect the speed of generating results to a certain extent.

    And it is also possible to call multiple models simultaneously, which will also affect real-time performance. Finally, in terms of the privacy and security of input images, further upgrade protection is needed.

    MSRA Veteran Goes Out

    The results of this research come from the team of Microsoft Research Asia. The corresponding author is Duan Nan.

    He is the chief researcher of MSRA, the research manager of the Natural Language Computing Group, a part-time doctoral supervisor of the University of Science and Technology of China, a part-time professor of Tianjin University, and an outstanding member of CCF.

    Mainly engaged in research on natural language processing, code intelligence, multimodal intelligence, machine reasoning, etc. He joined MSRA in 2006 and has served for more than 16 years.

    The first author, Chenfei Wu, is also a senior researcher. According to LinkedIn information, he joined Microsoft in 2012 and has worked for 11 years. He is currently a software engineer.

    Read Also: OnePlus Ace 2V Launched With Dimensity 9000, Starts at $330

    Do not forget to follow us on our Facebook group and page to keep you always aware of the latest advances, News, Updates, review, and giveaway on smartphones, tablets, gadgets, and more from the technology world of the future.

    Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

    Related Posts

    Glamee Dice 6000 vs. Glamee Mini Disposable Vapes Comparison Review

    Get the WLtoys A959-B 4WD RC Car at €53.93 From TOMTOP in Flash Sale

    Get Ready to Ride with the HAPPYRUN TANK G60 Electric Bike at €1089.99

    Leave A Reply Cancel Reply

    You must be logged in to post a comment.

    Top Brand
    Top Brand
    • Popular
    • E-CIGAR REVIEW
    • XIAOMI
    August 28, 2016

    ILIFE V1 Robotic Vacuum Cleaner Giveaway From IGeekphone Sponsored By Gearbest

    June 24, 2016

    Xiaomi MIBand 2 Giveaway

    March 1, 2018

    NOMU M6 Rugged Phone Design, Specs, Review, Release Date and More

    September 11, 2016

    Amazfit SmartWatch application in english

    September 26, 2023

    Power and Style in the Palm of Your Hand… Voopoo Drag M100S Review (Video)

    September 6, 2023

    The “CUTEST” Mini disposable ON THE MARKET!? Check the Dragbar BF600 Review (Video)

    September 1, 2023

    Lost at Disposable Vapes? So… Get a LOST VAPE! LOST VAPE OB5500 Review (Video)

    August 24, 2023

    COME & SEE… The DRAGBARs Disposable with LCD screen!!! 3500 Puffs and with Metallic Base. Dragbar B3500.Video Review

    September 29, 2023

    MIUI 15 Special Edition for Redmi K60 Pro and Xiaomi 13 Ultra

    September 28, 2023

    XGIMI Play 3 PTZ Projector on Sale: 400 CVIA Lumens, 1,999 Yuan

    September 27, 2023

    Xiaomi 14 Pro mobile phones have passed 3C certification to enter the network and support 120W fast charge

    September 27, 2023

    Xiaomi 13T/Pro series mobile phone released overseas: equipped with 144Hz OLED straight screen and Leica triple camera

    New Arrivals
    • Aspire Fluffi Pod System Kit Aspire Fluffi Pod System Kit
    • Aspire Gotek Pro Pod System Kit Aspire Gotek Pro Pod System Kit
    • Huawei FreeBuds Lite TWS Earphone Huawei FreeBuds Lite TWS Earphone
    • Snailhome Massage Reclining Office Chair Snailhome Massage Reclining Office Chair
    • Bakeey T9 bluetooth Headphone Bakeey T9 bluetooth Headphone
    • Bakeey D019 Wireless bluetooth 5.0 TWS Earphone Bakeey D019 Wireless bluetooth 5.0 TWS Earphone
    • ASPIRE CYBER X POD SYSTEM ASPIRE CYBER X POD SYSTEM
    • KUZ 9000 DISPOSABLE KUZ 9000 DISPOSABLE
    • Aspire Raga Aio Pod System Kit Aspire Raga Aio Pod System Kit
    TOP BRAND
    About

    Igeekphone.com provides the first global tech news and reviews about smartphone, vapes, e-cigar, smart home, 3D printers, e-bike,tablets, RC drones, VR headset, and other accessories. It's the best platform to improve your brand and product.
    Contact us: info@igeekphone.com
    *Check Our Privacy Policy Here.*
    Note: *Right now we have US editor and EU editors for review, especially for Amazon US and EU.*
    *Shop and Compare Price Here*
    PoE trade

    Deals
    September 29, 2023

    Smoant Pasito Mini vs. Smoant Charon Baby Plus Vape: Comparison Review

    September 29, 2023

    Get the WLtoys A959-B 4WD RC Car at €53.93 From TOMTOP in Flash Sale

    September 29, 2023

    Get Ready to Ride with the HAPPYRUN TANK G60 Electric Bike at €1089.99

    September 29, 2023

    Grab the Bisofice K6 Laser Engraver & Logo Marker Machine at €99.00 with FREE SHIPPING

    MIUI 14
    September 26, 2023

    How to Download Xiaomi 12 cupid MIUI 14 ROM and update

    NEWS
    September 26, 2023

    How to Download Redmi Pad SE xun MIUI 14 ROM Download and Update

    Copyright © 2023 igeekphone

    Type above and press Enter to search. Press Esc to cancel.