You're right. Words are just symbols. We make an association between those symbols, which are a stimulus, to ideas and objects, abstract and concrete. We hear or read a word or a part of speech and we make the connection between the stimulus and the thought it conjures. Everyone has different connotations for language.

But even so, I believe all knowledge stems from language.