Image Editing from Text Instructions with AI!

Good morning fellow AI enthusiast! This week's iteration focuses on a new image editing model called InstructPix2Pix, which, as its name says, translates a picture into another one following text instructions. We also welcome our new ethicist, Auxane, this iteration commenting on generative AI! We hope you enjoy it.

Receive the weekly digest right in your emails ??

1?? Image Editing from Text Instructions with AI!

We know that AI can generate images; now, let’s edit them!

This new model called InstructPix2Pix does precisely that; it edits an image following a text-based instruction given by the user. Just look at those amazing results… and that is not from OpenAI or google with an infinite budget.

It is a recent publication from Tim Brooks and collaborators at the University of California, including prof. Alexei A. Efros, a well-known figure in the computer vision industry. As you can see, the results are just incredible.

Feel free to try it yourself, as they shared a demo of the model on huggingface, which you can use for free!

What’s even more incredible is how they achieved those results. Let’s dive into it! (or learn more in the article)

Join the discussion with fellow AI enthusiasts ??

2?? AI Ethics with Auxane Boch

But first, let's introduce Auxane!

Auxane is a researcher at?iuvenal research, a?Doctoral Candidate at the School of Management of the Technical University of Munich (TUM), and a research associate?at the TUM Institute for Ethics in Artificial Intelligence (IEAI).?

What a resume!

After studying psychology in France (B.Sc.), and cyberpsychology in Ireland (M.Sc.), her current research focuses on accountability for AI systems, social robotics, video games and digital education. Auxane also?works closely with the Women in AI group on the educational side, and is a Women in Games ambassador.

She is excited to join us for the ethics segment of those newsletters, and is always happy to chit-chat if you want to reach out. You can contact her on her linkedin, or use her email address ([email protected])!

Now for Auxane's ethics bite...

The use of AI to modify images brings with it a lot of exciting fantastic perspectives, but also?a lot of legal and ethical questions. Let’s dive into the major points to keep in mind!?If we look closer into the use of AI for digital arts, two school of arguments face each other; the first argues that the use of AI will impede creativity, while the other assert?that AI is only but a tool and facilitator. The truth, as usual, probably lies in the middle. But, when considering digital arts and creations, another question arises: what about the legal aspect? If AI is used to create a piece, does it legally belong to the AI, to the entity who created the AI,?or to the human who used the AI? Those questions are not clearly answered yet, and arguments could be made from each perspective. Finally, I think it’s important to see the usefulness of such tools as a strong positive point to support the evolution of the technology. Yes, it can be used for deep fakes and sometimes maleficent purposes, but with the right frame of use, it can also support architects or landscapers to project what a building or a city will look like with their proposed work, or?by historians and teacher to generate accurate and detailed reproductions of old cities. As per usual in technology and ethics, the problem is rarely the tool, but the way we use it.?

Segment by Auxane Boch

We are extremely grateful that?the newsletter?is now read by over 11'000+ incredible human beings counting our email list and LinkedIn subscribers. Feel free to reach out [email protected]?with any questions or details on sponsorships. Feel free to follow our newsletter at Towards AI, sharing the most exciting news, learning resources, articles, and memes from our Discord community weekly.

Thank you for reading and we wish you a fantastic week! Be sure to have?enough rest and sleep!

We will see you next week with another amazing paper!

Louis