Only after I _ home, did I remember my doctor's...

Answer: 
went
Only after I _ home, did I remember my doctor's appointment.