Across a range of domains in psychology different theories assume different mental representations of knowledge. For example, in the literature on category-based inductive reasoning, certain theories (e.g., Rogers & McClelland, 2004; Sloutsky & Fisher, 2008) assume that the knowledge upon which inductive inferences are based is associative, whereas others (e.g., Heit & Rubinstein, 1994; Kemp & Tenenbaum, 2009; Osherson, Smith, Wilkie, López, & Shafir, 1990) assume that knowledge is structured. In this article we investigate whether associative and structured knowledge underlie inductive reasoning to different degrees under different processing conditions. We develop a measure of knowledge about the degree of association between categories and show that it dissociates from measures of structured knowledge. In Experiment 1 participants rated the strength of inductive arguments whose categories were either taxonomically or causally related. A measure of associative strength predicted reasoning when people had to respond fast, whereas causal and taxonomic knowledge explained inference strength when people responded slowly. In Experiment 2, we also manipulated whether the causal link between the categories was predictive or diagnostic. Participants preferred predictive to diagnostic arguments except when they responded under cognitive load. In Experiment 3, using an open-ended induction paradigm, people generated and evaluated their own conclusion categories. Inductive strength was predicted by associative strength under heavy cognitive load, whereas an index of structured knowledge was more predictive of inductive strength under minimal cognitive load. Together these results suggest that associative and structured models of reasoning apply best under different processing conditions and that the application of structured knowledge in reasoning is often effortful.