Well, firstly, don't listen to conventional wisdom. An inverted yield curve can occur as a result of many different things (for example, in the UK the yield curve has been inverted since 2004 or smth; that doesn't mean it indicated any sort of a recession). Secondly, you're right the shape of the curve is a complex interaction of a few volatile factors, one of which (default/inflation risk premium) you have mentioned. That's precisely the reason that one shouldn't listen to simplistic and overly general conventional wisdom. You should always try to understand the main drivers yourself and have an idea of what matters the most at any given time.
In principle, there are three things that can determine the shape of the curve: rate expectations, risk premium and convexity. If you want to understand more, read Antti Ilmanen "Understanding the Yield Curve" series of papers.