How do you create consistent and coherent icon sets?

Consider the metaphor you're using to symbolize your icon. Many icons have established understanding (i.e. a magnifying glass means search) so it's best to reuse those for consistency from a user's POV. If you are designing a net new icon, do research and testing to make sure that the symbol makes sense for your users. If it's ambiguous, consider adding text to accompany the icon (+ brownie points for being extra accessible!)

Less is more. In my experience, the best thing one can do is to not design yet another icon. Do your best to simplify and abstract the concept you are trying to portray to a level where you might find an established universal icon. Maybe changing the color will do the magic to go with your brand? Off-course, there will be exceptions to this.

Your icons are an extension of your language, voice, and content model. Be consistent and support the core idea, not just finding a rock climber graphic to complement the word “challenge.” Often there may be status icons that relate to different types of documents or processes. You can use multiple icon elements to show the unique process and use the status icon as a badge. I.e. an alert badge on a document icon vs an alert badge on an event icon. tl;dr - Don’t just complement words, build a language for your content.

I echo a lot of what the other contributors state; you need to generate something simple that tells a story within that small icon. Metaphor is a good storytelling tool, and less definitely is more. As Le Corbusier said “form follows function”. No need to over-embellishment if it doesn’t add to its purpose throw it out. Iconography needs to have a good foundation. A good area of communications study of note is “Semiotics”. A field dedicated to decoding visual contexts/symbolism within image and text. Using this in practise may give you interesting symbolism for your icons; you want things to be super simple. Like a pointer for a mouse or a pair of glasses for search functionality; you want signifiers that connote your intended message.

I nearly went mad developing thousands of icons for a CAD application many years ago. Trying to visualise a symbol for a process is hard enough when it's 'turn something on/off', but when it's complex instruction like 'choose nurb point to edit' and numerous variations, it can scramble your brain.

Not sure I 100% agree with just following icon styles of existing guidelines. Icons can help maintain and persist the brand of a given company or application. By simply using the set of icons that come with Material Design you are not allowing your brand to be unique and differentiate itself from the other brands that are using those same icons. I feel like this is conflating using the actual icons vs. using the principles for creating icons defined by the Guidelines.

A grid helps for at least two reasons: 1. It helps to ensure that you can visually balance the size and weight of your icon set, as different icons need to be sized up and down within the grid in order to optically match alignment or size of others. Line weight and size of details can be kept consistent by aligning elements to whole unit increments. 2. It can help your icons to scale up and down more elegantly, by aligning elements to whole unit values that can be multiplied and divided in convenient ways. Choosing numbers for sizes of details that have many factors (like 12 and 24) allow those elements to scale and to stay aligned to whole unit values (2,3,4,6,8)… keeping those details crisp no matter the display density.

It's essential to probe user comprehension of your icons and how they would interact with them, both in isolation and in real-life scenarios. To make sure you're setting your product up for success, deepen your understanding of your users' mental models by assessing criteria such as findability (are users able to quickly locate them in their actual placement?); recognition (do users understand what the images represent?); information scent (can users infer what action they will be performing when interacting with them?); and attractiveness (are they engaging and visually appealing to users?).

One benefit of using an icon font is that it can allow icons to easily flow inline with text, and to scale with text using the same scaling units. This may not be desirable in all instances, as using text size units and a font may result in somewhat unpredictable or undesirable outcomes if someone changes their default font size preferences at their OS or browser level. They can also be somewhat harder to use and access in common design tools like Figma, and may require special plugins or references in order to find and use the right icon. If you can create or use an icon set that offers both a font and SVG set, you can get the best of both worlds. Be careful that you don’t take a performance hit in your product loading both though!

Outlining your font and ensuring any strokes are outlined will mitigate scaling issues. This way you can use whatever font you want (as long as you have licence to use)