I don't remember the first FF game that I've played but I also remember playing FF1 on gameboy but not sure if it's the first FF game. I think it's the FF 3 on the PSP, I also did remember playing it and for what I remember it's the first FF game that I've saw and played with.
Gameboy Advance, that brings me back memories! I think the first time I played it was there too, and it was already an awesome game. It has evolved a lot, but it was already excellent at the time of the first release.