I got a Gigabeat F40 and I figured out how to delete pictures. It's unintuitive and convoluted, but it works.
You need to have the Gigabeat connected to the computer, then access the Gigabeat as you would a hard disk in My Computer. Enter the "picture" folder and you will see a bunch of files. For every jpg you place in the Gigabeat, there will be three additional GBP files created. These files are used as thumbnails when you use the Gigabeat alone. Even if you delete the image files in the Gigabeat photo browser from the Gigabeat room program, the GBP files will remain and show up as normal images when you use the Gigibeat alone.
I have a jpg of a Nutella jar called "nutella.jpg" which I transfered to my Gigabeat. In viewing the file via My Computer, it was renamed "nutella0601271502.jpg" and it also created "nutella0601271502_1.GBP", "nutella0601271502_2.GBP" and "nutella0601271502_3.GBP". If I wanted to remove the Nutella image completely, I'd need to remove ALL FOUR files. I tested this with some other photos I had put on my Gigabeat and I'm happy to say that it worked without harming my Gigabeat.
Good thing I found this out too because I had put a lot of furry porn on my Gigabeat and it would suck if they stuck there for prosperity.