Why orange? I don't know. Why not orange? I like orange, in plants anyway. Not so crazy about it for curtains or appliances and so forth. Van Gogh is said to have called orange "the color of insanity" (and he would know); a few little dots of insanity here and there is good. Large, uninterrupted fields of insanity is more problematic, though.
Very few of these plants have flowers which are exclusively orange. In some cases, environmental conditions influence bloom color, though usually not a lot, and in a lot more cases, non-orange varieties, as well as orange varieties, exist. If the flower color is particularly important to you, and apparently it is or else why would you be here, you should buy plants which are already in bloom or get the seller to confirm the bloom color before you buy.
For the recommendations, I like Aeschynanthus, Anthurium, and Clivia, with Strelitzia as an honorable mention.
Aeschynanthus is a little bit difficult: mine bloomed in 2008 and 2010 but not in 2009 (probably because of the move), and it's still a little lopsided and gangly looking. This is probably because it started out as a bunch of leaf cuttings, and is still working on filling itself in (even two and a half years after the leaf cuttings were taken). Still, though, the flowers are very bright orange, they last for a pretty long time, and when the plant is happy, there are a lot of them. Plus for a gesneriad, it's not terribly difficult.
Clivia I like even better than Aeschynanthus, though so far that's as a foliage plant, not a bloomer. One of my two Clivias is too young/small still to be blooming, so I don't mind it. The other, which had yellow flowers when purchased, has not bloomed for me, probably because I haven't given it the cold temperatures it wants during the winter. I'm not under the impression that it's particularly hard to get it to bloom, just that I haven't been able to provide the conditions it wants. Maybe this winter. In any case, it's a nice foliage plant, and the flowers are very pretty, even if I haven't seen many in person.
Anthurium's inclusion on the recommend list should surprise no one, since I'm an unabashed fan of the plant, have had all kinds of great experiences with them, and encourage others to buy one on very little provocation.
Lots of good candidates for the anti-recommend, but I'll go with Cuphea ignea. I have very little experience with them, having only bought one a couple months ago, but what experience there has been has been pretty uniformly negative. All the flowers dropped within a couple days, about a third of the foliage followed closely thereafter, and we're only just now getting around to the production of new foliage (but not, so far, new flowers: I will be very surprised if I can convince the plant to flower for me again). I won't give up on the plant unless it gives up on me, but so far it's looking like one of those outdoor plants that needs to stay outdoors. I could pretty much say the same for Hibiscus rosa-sinensis, but at least with Hibiscus you have a chance of seeing some pretty incredible flowers.
Abutilon cvv. (some)
Achimenes cvv. (a few)
Aloe 'Doran Black' (pink-orange)
Aloe aristata (pink-orange)
Aloe brevifolia (red-orange)
Aloe ferox (depends on specimen but usually some orange)
Aloe maculata/saponaria (pink-orange)
Aloe striata (red-orange)
Aloe variegata (red-orange)
quite a few other Aloe spp.; usually red-orange or pink-orange
Begonia cvv., esp. Riegers, non-stops (some)
Bougainvillea cvv. (a few)
Calathea crocata (uncommon indoors)
Calceolaria cvv. (some cvv.)
Cattleya and Catt. alliance orchids (some)
Chamaedorea elegans (though not particularly pretty, and only orange in strong light)
Chamaedorea metallica (though not particularly pretty, and only orange in strong light)
Chamaedorea seifrizii (though not particularly pretty, and only orange in strong light)
Columnia cvv. (most?)
Crossandra infundibuliformis (most; there are a few yellow, pink or peach ones around)
Echeveria cvv. (some)
Episcia cvv. (some?; red-orange)
Euphorbia milii? (a few; more peach than orange)
Gasteria cvv. (most/all?)
Gazania cvv. (not particularly good indoors; most are not orange)
Gerbera cvv. (some)
Gloriosa spp. (some)
Guzmania cvv. (some)
Gynura aurantiaca (though not particularly pretty, and definitely smelly; yellow-orange)
Hippeastrum cvv. (a few)
Impatiens hawkeri cvv. (some)
Ixora chinensis and I. coccinea cvv. (some)
Kohleria cvv. (some)
Lantana cvv. (a few; not a particularly good houseplant in my experience)
Masdevallia cvv. (some; yellow-orange)
Medinilla cvv. (some)
Mokara cvv. (a few)
Oncidium alliance orchids (a few)
Parodia microsperma (at least one variety; they're usually yellow and occasionally red)
Pelargonium x hortorum cvv. (a few; red-orange)
Schlumbergera truncata cvv. (some; pale orange / peach)
Senecio jacobsenii (though not particularly pretty)
Stapelia spp. (some spp.; orange-brown or peach or both)
Thunbergia alata (also white, yellow)
Tillandsia spp. (a few; tend to be more peach than orange)
Vriesea cvv. (uncommon; more red-orange)
Anybody have any suggestions for plants I left off the list?