Run 'vmstat 1' in a different terminal on the RPi while running stuff, and paste the first dozen lines? (You can just CTRL-C vmstat after this)
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 0 124928 16680 61392 0 0 0 0 4120 552 3 40 57 0 0 0 0 124896 16680 61392 0 0 0 0 4315 542 4 41 55 0 1 0 0 124864 16680 61392 0 0 0 0 4220 549 3 40 57 0 0 0 0 124748 16680 61392 0 0 0 0 4079 544 3 39 58 0
The problem occoured at the second line