DT1 uses sign-magnitude representation to allow for both positive and negative detunes. Values 1-3 are positive detunes, while 5-7 are negative. Setting one operator's DT1 to 3 and the other to 7 will produce a more powerful effect.
When the VSYNC interrupt occurs, the system jumps through the vector located at $0314, which points to the kernal interrupt handler (unless the vector has been modified). This routine performs several functions (polling the keyboard, updating the system timer, etc.). Before returning, the routine clears the interrupt status in ISR to prevent the interrupt from immediately triggering again. To properly wait for VSYNC, you will need to modify the vector at $0314 to point to your own interrupt handler. A custom handler will likely set a value somewhere in memory indicating that an interrupt has occured, and then jump to the original handler.
In order to change to 40x30 mode, you should use the screen_set_mode kernal routine. In addition to changing the screen size, this routine sets some variables in the kernal that are used to determine when to scroll the screen.