We construct and analyze a rate-based neural network model in which self-interacting units represent clusters of neurons with strong local connectivity and random interunit connections reflect long-range interactions. When sufficiently strong, the self-interactions make the individual units bistable. Simulation results, mean-field calculations, and stability analysis reveal the different dynamic regimes of this network and identify the locations in parameter space of its phase transitions. We identify an interesting dynamical regime exhibiting transient but long-lived chaotic activity that combines features of chaotic and multiple fixed-point attractors.