Something has to keep the audience interested in watching an animation, and that something is usually the story. A cool story with great characters can overcome any TV budget (Chou! Kuseninarisou is one of my favorite TV comedies, yet most of it is told with held cels with mouth animations).
Clever writing can even set things up so that fewer characters and sets are needed (a live-action example of this is the hilarious Red Dwarf television series, involving four guys and a sentient computer trapped in a huge spaceship millions of light-years away from Earth). Fewer characters/sets could mean that much more money spent on each character/set, improving the overall quality of the animation. (In my opinion).
The director's next in the chain. It doesn't matter how good the story and crew are -- without a good leader, everything will be lost in confusion, and the whole will not exceed the sum of its parts. (In my opinion).
Work out every storytelling problem at this stage. Use them to visualize your final animation. Once completed, they will act as your roadmap throughout your entire project.
The "making-of" book for the Ghost in the Shell Playstation game includes the storyboards drawn by the director for the spectacular opening animation. The detailed storyboards look almost exactly like the finished piece (the biggest difference being that Motoko Kusanagi wears her visor down during most of the finished animation).
Good art direction not only makes every still frame look good, but it also means a consistent and distinct "look" for the entire piece. Art direction, or its absence, will be felt throughout an entire animation.
Kiki's Delivery Service looks nothing like
Ninja Scroll, which in turn looks nothing like Memories, which itself has three separately animated pieces that look nothing like each other.
From the gentle look of Kiki's Delivery Service, to the ornate feel of Shamanic Princess, to the stylized realism of Spriggan, character designs are something the audience will view for as long as a character exists on the screen.