"It's up to you to root yourself into the ground and understand who you are.” - Nicole Richie

Checkout this week's new episode of the #NoLimitsPodcast featuring fashion designer and founder, Nicole Richie:

On today’s episode, how Nicole Richie has built a fashion empire. The Los Angeles native gained notoriety at an early age, being the daughter music legend, Lionel Richie and traipsing around the country alongside Paris Hilton on The Simple Life. I watched that. Did you? But Nicole has proven herself to be so much more. She grew up with a passion for design, turning her dad’s excess costume fabric into her competition ice-skating outfits. And although she abandoned her dreams of being an Olympic ice-skater, yes, she wanted to be an Olympic ice-skater, she kept her design aspirations strong and eventually turned them into a business. House of Harlow 1960, now ten years old, the company has carved out its own niche in the fashion world. We caught up with Nicole at her office in LA to tell us how she did it. 

R: Nicole Richie, welcome to No Limits!

N: I’m so excited!

R: I’m so excited. Thank you for having us to your office.

N: Thank you so much for coming.

R: It's beautiful here. And the first thing you notice when you walk in is the smell. It's incredible.

N: Yes, it’s me.

R: What is it?

N: Oh the room. The smell of the room is House of Harlow candles. I did three candles. So, that is the St. James. We burn a lot of candles in here.

R: Well, it smells great. And I love also obviously all the beautiful clothing and shoes. And, of course, we noticed the tape collection.

N: That's my most important element here. Are the vintage—did you see the All-4-One cassette tape?

R: Yes. Obviously.

N: That's very important.

R: All-4-One. Really key.

T: That was my first concert ever. 

N: Also Beaches. Let's not sleep on a Beaches cassette because somebody just recently told me in this office. She's 26 and she says, “What's Beaches?”

R: This is what freaks me out.

N: “What's Beaches?” Are you insane? That's crazy. I said, “What do you mean? Wind beneath My Wings?” She responds, ‘No, I don't know what that… I don't know what that is.’

R: This is a scary moment in our lives.

N: It's tough times.

R: You're 36. I know this because I’m 36 and our birthdays are a week apart.

N: September 21st.

R: September 28th.

N: Oh my gosh.

R: So I'm Libra, you’re a Virgo.

N: Yep.

R: But yeah I was doing some research and I was like, “Wow, our birthdays.”

N: We can have joint birthday parties, except we're not the same sign so, I don't know how that's going to work out.

R: Well, we're really happy to be here thank you for having us to your offices. I look at your life and everything that you've built at this point at age 36.

N: Yay!

R: Do you feel that way?

N: I do. Yeah, I do. You know what? It's been a not long process, but it has been quite a process getting to where I am and I'm by no means finished. But this has been definitely a journey and although there are, of course, points of it where I'm thinking, “Oh, time has flown by so fast,” there are other parts where I'm like, “Oh my gosh. I was a totally different person.” When you think about building a brand, especially being one person at 26 and another person at 36, you're contemplating, “Oh my gosh. I totally forgot about that. I was approaching it in such a different way.”

R: So, House of Harlow started in 2008 when you were 26.

N: Yes.

R: What was the vision back then and what is it today?

N: Well, when I started I decided that I wanted to do jewelry. It started off as just jewelry and I was also having a baby. My entire life was about to change and I was invested, but also being 26, “Oh, I'll see how this works.” It just started off as I did a small licensing deal and it was just kind of a moment for me to see if it was something that I was even into. But thinking about the future in your 20s is just something that rarely happens, right? I didn't even have the mindset of, “Okay, where do I want this? How do I want to expand? Where do I see myself in 10 years?” I wasn't asking myself those types of questions. I was really just testing the waters and seeing if creatively it was something that spoke to me. I had been making costume jewelry for my friends and for myself. I do that all the time. I was traveling with my husband and in every city that we're in, I have nothing to do during the day so, I would go to little kiosks and get anything that was inspiring me and make jewelry out of it and bring it home. That was just what I liked to do. But, of course, building a business around it is a totally different thing. So it started out as jewelry. Two years later, I expanded into apparel and footwear and then, grew from there.

R: You talked about starting early. I read that you would design your ice skating costumes.

N: That's right. Yes.

R: You’d always been into design.

N: So, Nancy Kerrigan wore a Vera Wang figure skating dress to the Olympics. I don't know if you guys remember that, but it happened and it was my first introduction really to any sort of luxury brand. That was my life. I was very swept up in Nancy Kerrigan. She was everything to me. The Tonya Harding and Nancy Kerrigan drama turned Culver City ice skating rink upside down. I was in it. So, my dad had two costume designers: a woman named Edna and a man named Bill Whitten and they were making my figure skating outfits- all of my competition outfits. And I would say what I wanted, “Oh, I want it to look like this.” Edna and Bill have known me since I was three-years-old and they would always make me outfits out of my dad's excess costume fabrics. So finally, when I was skating and when competing, I was saying, “Oh, I want it to look like this and this,” and she said to me, “Why don't you just come downtown and start designing and making them yourself?” And it was so exciting for me and I spent time downtown. If you are at all a control freak there is this magical element of having something in your mind and then making it come to life. That's so exciting. Once you understand what the processes of doing that, it's really hard to go back.

R: Are you a control freak?

N: I'm a Virgo. So, a little bit.

R: Back to being a kid and growing up among a lot of celebrity. Were you thinking at the time, “I want to be an Olympic ice skater?”

N: Yeah obviously.

R: So you were on the path to that.

N: Yes.

R: Was there some moment that took you off that path?

N: I was training in Lake Arrowhead every summer. I went to Switzerland every November with my coach Vonn. I was in it. I was competing and I was in it. And also, the older I got - I was a teenager, and wanted to live that life too. When I was 18, my parents said, “Well, you have to decide. Do you want to go to college or do you want to really go down this road?” And so, I chose college.

R: Were you worried about that decision? We had Aly Raisman on the show not that long ago and she just talked about the level of commitment. I mean, I think it's almost impossible to even perceive how much dedication you have to have if you're on the way to the Olympics.

N: Well, I wasn't even in school. I was in tutorial school until 11th grade. 8th through 10th grade, I was going to school four hours a day in Malibu and I was spending the rest of my time in Culver City just training. I, by the way, was still nowhere near getting to the Olympics. So it really takes over your entire life and it just got to the point where I couldn't do both. And, you know, my teenage wild years were just calling me and I had to dive in.

R: Do you ever feel grateful that those teenage wild years happened pre-Instagram?

N: I think about it every day. I'm so happy.

R: I was thinking about The Simple Life a little bit when I was getting ready for this conversation and just thinking how different it was back then to watch you and if there had been a Snapchat, we would see everything.

N: I do think that these teenagers are very mature because they are so aware of what they're putting out into the world. I was not. I don't even know if I would have the self-control to be a teenager when all of that existed.

R: Well, they're thinking of it as branding today.

N: Right. Right. They're saying, “No, I'm forming this into a business.” Which I also wasn't thinking about.

R: At what point did it occur to you that you wanted to start thinking of yourself as a brand and as a business?

N: Really when I started House of Harlow and I didn't even think of it as myself. I didn't have the brand under my name. After my first season of doing jewelry I thought, “Okay, this is definitely what I want to do.” And when you start a business, there are so many different ways you can go about building your business and a lot of different people in your ear. I've definitely gone down multiple roads of how to build this brand. There are so many parts of this business that I would have never understood. It's time and experience. That's just what has to happen. And I just knew. I thought, “This is what I want to do.”

R: It's great. And you don't even know along the way if you're making those right decisions or not. You talk about all those voices in your head.

N: You have you have no idea.

R: Not your own voices, but the people who are feeding into that and I'm sure there was pressure when you started. And it's an interesting choice to go with “House of Harlow” as the name, because there were probably people saying, “Name it Nicole Richie.”

N: Yes, yes. There are endless people in your ear and also because it was a license business, my licensing partners had a big say also. At 26, I couldn't have told you what my key stores were or what my key retailers that I wanted to be in were. We were just really throwing everything out there and leaving the decisions up to my partners. Which does work for some people, but it's those decision making moments where you think, “Okay. Well, am I the type of person to care about where my stuff goes? Yes, I am actually so, I'm going to take things this way,” and it's a very gradual process. Each stage gives you an opportunity to understand yourself a little bit more and it's a lot of looking inward and understanding what's important to you. What are your goals? What do you want out of this? How do you want to spend your time and understanding the value in your time. My time is the most valuable thing I have, especially now. I have two kids that need me. My time is everything to me so, how I spend it I'm very specific about.

R: Two kids. You're also a philanthropist with baby2baby.

N: Yes.

R: How do you structure your time?

N: How do I structure it?

R: Do you wake up at the same time every day?

N: I do. Yes. I wake up at 5:30 every day. No one else is awake. It's just me. And that's just my time to be by myself. I really do try and I know myself, I know that I can get overwhelmed pretty easily so, I do have to put limits on myself and what I can do. Because it's extremely important for me to come, be somewhere and be 100 percent there. But when you're excited about something and it's a lot of work- I'm not going to lie, but it's so exciting and it just gives me energy. It just makes me not tired. When I know that I'm doing something and it feels right, it feels truthful and aligned with who I want to be, then- I don't know, it just gives me a different surge of energy.

R: So you wake up at 5:30.

N: Wake up at five thirty and do nothing. Lay in bed.

R: Do you scroll through e-mails at that point?

N: No no no no. There's no e-mails. That I wait for. I'll read or I'll just lay there. Walk around the house alone. It's very peaceful and quiet. The birds start chirping at around 5:15 a.m. so, it's nice. I feel like I'm waking up with the world, I feel like I'm waking up with nature. And then, my kids wake up at about 6:30 and that is a wild race every single time. That's the most exhausting part, getting them to school on time. Everything else actually is kind of easy. The drives downtown, I can handle all that.

R: Are you an on time family?

N: I am, yes. My daughter is, my son doesn't care and my husband doesn't really. They're fine being 15-20 minutes late. My parents are late and I hate it so much. I will never be late. I also get anxiety feeling like people are waiting on me. I like to be somewhere early so that I can take my time.

R: I feel like I'm always rushing and my parents totally taught me that. Taylor's laughing because she knows that I am always rushing and my parents taught me that because we were always like racing to get to school on time every single day.

N: It's a race every single day. It's like the Olympics. It really is. I feel like it is a race every single morning.

R: When you think about your life, I would imagine that along the way you have been underestimated.

N: Sure.

R: Did you feel that way from a young age? Was that something that started when you started getting into the business side of things?

N: Yeah, I mean there is a certain level always of feeling like you have to prove yourself and then, you get to a point where right now I feel- because this brand is has been around for nine years, these nine years have been a constant process of letting go of everything that's not authentic to who I am and that's not easy. And I do feel like I'm now at a place where I can walk into a room with any man in this industry and I can have any wholesale meeting and I feel confident in what I'm saying because it's the truth and nobody can question your inner truth. It's really about sticking to who you are.

R: I'm shaking my head in absolute agreement with you and I also think the way you've described it there is this progression. And I think it happens for all of us at some point you start something earlier in life and, of course, there's going to be the uncertainty and you're going to have the various people factions telling you what to do. And early on it's hard not to listen to that to some extent. Sometimes you're going to do things along the way that you're sort of half in half out.

 N: Yes.

 R: And then there comes a point where you know you have the vision, you really strongly believe in it you know it's resonating with people. Where was that turning point for you?

 N: Well building a business is something I had to learn. I am a creative at heart. It needs to mean something to me. So I decided to pull everything back. A decision to take all of my apparel out of wholesale because I just wanted to speak directly to my customer and that was a decision that I was advised for a long time not to do. But it got to a place where it was pulling at me. And it was just something that I wanted to do when I wanted to speak directly to the girl. Designing is like storytelling and it was really hard for me to ask somebody else to tell my story and to speak to the girl who I felt so badly that I knew.

 R: Totally. I mean, there's so much junk out there.

 N: Yeah.

 R: There really is. For anybody who shops or in the store and you think, “Why is there such junk?” Well, there's junk because people are bending to the will of their investors.

 N: Yes.

R: They're making choices to cut corners.

 N: Yes. And to appeal to the masses.

 R: Right, and to appeal to the masses. But even the things that are meant to appeal to the masses, the corners have been cut so heavily, especially in apparel and retail, and that was somewhat an outcome of the Great Recession- which is basically the time when you're creating this business, the height of the financial crisis. And I often wonder if we're going to see a swing back to more quality products. We're definitely seeing a swing back to people wanting to understand the genesis of their product, wanting to understand the people behind the product and they care more about it. The consumer is more conscious than ever before.

 N: Yeah. Listen, it's very easy to make whatever you want now, right? I was on the boardwalk of Venice Beach yesterday and you see 900 different versions of heart framed sunglasses and you're thinking, “Oh my gosh, I could... So, how much is this?” They said five dollars. I thought: “Okay great. Okay fine.”

 R: “I know I could make those for 30 cents and make an amazing margin on that.”

 N: Yeah, I do think that people do want quality things, but also if you're asking people to spend their own money what they really want to be connected to is the story and the reason behind it. And that's what I've found for my girl and you had said appealing to everyone and taking advice from your investors. You really have to be okay saying to yourself, “I'm not for everyone,” and some people are. I am not and that bleeds further than my business. That's also just who I am. My humor is very specific. I am a specific person. I have let go of trying to be everything to everyone. I do know that there is a customer and a girl that I am speaking to and so, I'm focusing on that and I feel like we're in this dance together and that feels better to me personally than asking everybody and begging everyone to just come and buy everything that I'm designing.

 R: What do you say to the entrepreneur who says, “well, that's all fine and good, but you're Nicole Richie, so it's much easier for you to do that than it would ever be for me”?

 N: I've done the opposite. I'm only saying this now because I have done it. And, you know what, If that's the road that you want to go down you should absolutely do it and see if it feels good to you. For me, it ripped me apart. I was no longer connected to a lot of different categories in my business and I didn't know what was out there in the world and it didn't feel like—I wasn't necessarily connected to everything that I was creating. And that didn't feel good for me.

 R: You wrote your bestselling book The Truth about Diamonds. Do you want to write more books?

 N: Oh my gosh. I don't know. I don't know if I'll ever write another book because I'm a reader now, so I don't know.

R: You’d rather read them than write them.

N: Yeah. I don't know. We'll see. Maybe—not today.

 R: How has your idea of success changed over the years?

 N: You know what, this brand had a very big role in that. Like I was saying to you, it's about my time. And I really want to be proud of everything I'm doing. I want to hold my head up high and say, “this is what I've created and this is what I'm doing and I'm doing this because it means something to me,” than just throwing something out there because somebody tells me to do it. And I've done that. I've done it many times. And it's fine. But you know what, you also have to go, and you have to talk about it. You have to talk about it. Even if you're not me, even if you're not on this podcast, or doing a talk show, you have to go and self-promote whatever you are putting out into the world. And so, I want to spend my time doing something that excites me and makes me feel good. And there's a lot that I want to do and plenty of it. And so, that's the only way that I want to spend my time.

 R: With baby2baby. Tell us a little bit about the work that you're doing with the organization.

 N: So my best friend. Her name is Kelly Sawyer and she's one of the presidents. She runs it with a woman named Nora. So, they give gently used items to families in need within the community, and raise money to get diapers and clothing, cribs, car seats, everything that mothers who are living under the poverty line need. You know, I wanted to really understand the business more, and so I was really taking a—I've been spending a lot of time with them and really diving in and understanding. It's more than just giving items to mothers. It's really understanding how this state and how our city is set up. So, you know, it took me asking questions of, “well, why is it this way?” or “why can't they?”—Like “what about this? What about this?” and really understanding that, you know, it's not easy for a mom who has two kids at home to take a bus that costs money to get to target, which is not in her community, or to a Costco to get diapers. I had no idea that diapers were taxed as a luxury item. And so they really are trying to change the way that our city is run. And in that, you know, it gave me an opportunity to say to myself, “OK how do I feel like I can make the biggest difference in this?” So of course. I go to the events and the gala and I donate our books and our clothes to kids. That's very easy. But spending time with these families and really understanding the concept of being born and raised basically understanding that you're never going to have a chance. That's really heartbreaking. And instead of just feeling bad for them, I really want to help them. And if you want to make a difference in the world, you can make the biggest difference by starting within your own community, by starting small, starting either with the schools in your neighborhood. But really building up your community can make such a difference in the world.

 R: I absolutely agree with that. And I also think, you know, oftentimes you think about a really big problem.

 N: Right.

 R: And so how do you solve a really big problem? You have to start.

 N: You have to start somewhere.

 R: And I think a lot of people get so wrapped up in solving this big problem. That they don't start. And it's just a matter of moving forward a little bit.

 N: Yes.

 R: What else is there Nicole? What are the other big things?

 N: Well I'm still trying to be Britney Spears’ backup dancer—working on that. Building this brand. I mean I really had to pull back for a minute and get focused and centered about who I am and what I'm saying to the world. And I also want my kids to see that I'm working hard to achieve what I want. And like you said, it's a very crowded space right now. There's a lot of people doing a lot of things. And I don't need to take up space just doing things just because.

 R: I feel it in my world. Even as a journalist, you're constantly reminded of what you're not doing on social media. You not at this particular thing—

 N: You could be more active on social media, you go to this party, and I say—oh my publicist is in the room. Kate Rosen, everybody!

 R: And Henry.

 N: And Henry my other publicist, who is a three pound cockapoo. And there have been many times I've said to—actually not Kate because she totally gets me. But I've said— I haven't been with Kate the whole time. But I'm like I don't need to be at this party—for what? There are so many people that are there and they're doing it. And I just don't need to do that. I would rather spend my time doing it this way. And that's just getting to know yourself. It's just time. And it's just listening and trusting your intuition and listening to yourself.

 R: Is there anything you did along the way that helped you get there? Were you deliberate about it or did it sort of evolve over time?

 N: I think that it really comes from making the mistakes of saying “yes” to a lot of things and not feeling great about them. Of, you know, answering to a department store of how they want something done a totally different way and me doing it and then it's out in the world and I can't fully stand behind it. And it doesn't work. Also it didn't work. Maybe if it worked I'd be saying something different, but it does not work for me. It's just not for me.

 R: Yeah, I think there's almost nothing worse than taking someone else's advice, going against your own instincts, and it doesn't work.

 N: Yeah.

 R: And then you just kick yourself. I knew this in my gut.

 N: I would rather something not work and I made that decision 100 percent. And then I can even stand behind the mistake.

R: Yeah.

N: But how foolish do you feel being like, “Oh, I only made that mistake because somebody else told me to do it.” That's like a double loss.

 R: Is that the toughest lesson?

 N: Yeah. Toughest lesson along the way is just kind of losing yourself, losing your signature, and your DNA, with all of the noise around you.

 R: The noise is so loud.

 N: But I don't even know if I could—even if I could go back and do it all over again, I really don't know if I could have just made those decisions in my 20s. I really feel like the only reason why I can so easily say, “yes, this is what I want to do” or “no, this is not what I want to do” is because of the nine years that I've been building this brand.

 R: If you could go back to 14 year old Nicole, what would you tell her?

 N: Stop over tweezing your eyebrows. Stop being an animal and lucky you that you get to spend time in school all day. I did not appreciate education at all. And I live and die for it now. And if someone could stick me in a school where I just learned for eight hours a day, I would be so happy and so lucky. That's the biggest part of my 14 year-old self that I'm really bummed about.

 R: Do you think she would listen to you?

 N: No.

R: I was thinking maybe because it's you, you would know how to get through to her.

 N: Mm. Maybe. I don't think so.

 R: What's the worst advice you received along the way?

 N: Worst advice. Well, this is not a business thing. But I was pregnant with my daughter and no one knew and it was maybe going to come out. And I got on the phone with this publicist who was on my team, but she wasn't my publicist, and she gave me a 15 minutes minute speech about something that Britney Spears did and two other people did. And so that's what I needed to do. And I got off the phone with her and I was like “this person has no—I don't know what we're talking about.” By the way, I am the biggest Britney Spears fan. You can be a fan of somebody all you want, but this has nothing to do with me. I'm not just doing something because other people did that. I have to stay true to who I am. I don't know what you're talking about.

 R: Can I ask what it was.

 N: Oh she was said, “you should you should just go out—” And I was 9 weeks pregnant. I wasn't even 12 weeks. And I thought, but my doctor, I haven’t even told my friends. I don't know what you're talking about. It was so long ago that I don’t fully remember.

 R: It was, “make a big a show of it right now.”

 N: She said, “Britney Spears did it this way” and I said, “but I'm not Brit Brit—what are we talking about?”

 R: I actually I think that—

 N: It was so separate from who I was and I was being thrown into this celebrity bucket. And she says, “you got it. You'll do it this way.” And I thought, “No. I'm not just a famous person. I'm a human being. And this is how I feel.

 R: It's bigger than what that moment is to your career.

 N: Yeah this is my life. This is my life.

 R: Completely 100 percent. Did you communicate that or did you just take it with a grain of salt?

 N: I didn’t really know her. I think I got off the phone with her and said, “I'm never talking to that woman again.” It wasn’t even bad. I was just like, “this is a waste of 12 minutes.” You will constantly be given the opportunity to lose yourself. Always. It's never ending. It doesn't just go away. You can't cut that many people out of your life and just assume that it's not going to happen. That is a part of life. It's up to you to root yourself into the ground and understand who you are. And if something doesn't feel aligned with you, then you politely just say, “no thank you” and move on.

 R: You're so grounded.

 N: I'm so grounded.

 R: You really are.

 N: I was joking. But thank you.

 R: No you really do. You come across as very grounded and very down to earth. And I have to say, back in the day, when I watched you on The Simple Life, maybe you didn’t think you were that person.

 N: Okay, I’ll take that. Any other thoughts you had of me?

 R: Love the outfits.

 N: Henry?

 R: To me, it's a really refreshing thing to see that people can change—or people can evolve is a better way of putting it.

 N: Yeah and you know what, I think there was this public perception of me that was, “you’re so all over the place” and whatever it was that you thought, I'm sure you don't want to say it on this podcast, but I can tell by the look in your eye that it wasn't the best. And then that puts pressure on you to prove yourself to be the opposite. Right?

 R: Right.

 N: And then you kind of—and I've experienced that even going into the fashion industry. Being from California is not what people wanted to hear ten years ago. You live in New York. Building a brand and being from California was just not—it’s different now. And then I have gotten to a place, multiple times, where I'm just think, “no, I'm actually just going to dive into who I am.” You know, I don't need to prove to be the opposite. I don't need to prove that I'm—”You thought I was wild and so I need to prove that I'm not wild” or “you thought I was dumb, so I need to prove how smart I am.” Yes I am from California and I love having fun and I love dancing and I love hip hop and I'm a good person, at the same time. You know? So, those moments where you feel like you have to prove yourself to be the opposite are just as important, but I'm also grateful for them because they've given me the opportunity to really just lean into who I am.

 R: Nicole Richie. Thank you so much.

 N: Thank you guys!

R: This was really fun.

N: It was so fun! Do you want to stay?

 R: Yeah.

 N: OK great I’ll dress you. We can try on clothes.

Thanks so much for listening! If you like this episode, please subscribe to "No Limits" to rate and review!

Follow me on Social Media:

• Twitter
• Facebook
• Instagram