Vector symbolic architectures (VSAs), such as holographic reduced representations (HRRs), can be understood as schemes for representing knowledge and implementing symbol manipulation in connectionist architectures. We characterize VSAs in terms of five basic operators: similarity, superposition, binding, unbinding, and permutation. We compare the different VSAs in the literature on space and time complexity and unbinding accuracy, and on qualitative differences in how their operators function. We find that binary spatter codes, and real-valued and frequency-domain HRRs compare favourably to other types of VSA. We discuss the advantages of VSAs over traditional connectionist and symbolic approaches. VSAs retain the expressive power of symbolic approaches with the added benefit of being lossy representations. The lossy nature of VSAs makes them easily scalable to large problems and suited to modelling biological memory. Whether VSAs can be applied to problem domains that cannot be trivially discretized into symbols remains an open question.