It seems to be a problem with vibration.
I had had my little scanner on the top of the computer, which lives in the hutch under the desk. Simply moving it to the solid window ledge and, for good measure, putting a small stack of books on top before each scan has transformed the results from hopeless to definitely OK.
|View from Caernarfon Castle -- summer 1987
Scanned from an old slide. Some slides scan much better than others. It really doesn't like dark photos, which is a bit of a bummer, but I can live with it.