I do not know the exact compiler flags, but you can de-assemble your *.o files and see if they contain VFP instructions. Use the arm-elf-objdump command for it, it should be next to your arm-elf-gcc command. It should show assembler code like FMUL FADD,FDIV in there. The registers of the VFP have names like s14 (single) or d5 (double) that should also give you an indication. If it shows calls to library functions instead (BL xxxx) then your toolchain is not generating VFP instructions.
As an example, this is part of some disassembled floating point code for me:
2c: e59f3094 ldr r3, [pc, #148] ; c8 <.text+0xc8>
30: e50b3014 str r3, [fp, #-20]
34: ed1b7a04 flds s14, [fp, #-16]
38: ed5b7a05 flds s15, [fp, #-20]
3c: ee677a27 fmuls s15, s14, s15
40: ed4b7a06 fsts s15, [fp, #-24]
44: ed5b7a06 flds s15, [fp, #-24]
48: eefd7ae7 ftosizs s15, s15
4c: ee172a90 fmrs r2, s15
50: ed5b7a06 flds s15, [fp, #-24]
54: eeb76ae7 fcvtds d6, s15
58: ed9f7b14 fldd d7, [pc, #80]
5c: ee366b47 fsubd d6, d6, d7
60: ed9f7b14 fldd d7, [pc, #80]
64: ee267b07 fmuld d7, d6, d7
68: eebd7bc7 ftosizd s14, d7
6c: ee173a10 fmrs r3, s14
Hope this helps.
-Tom