1) CORBA works with more than just Java.

EJB can use IIOP, but while

I can get an EJB client to talk to a CORBA server, I can't get a CORBA client
to talk to an EJB server.
2) EJB containers are typically much slower than custom CORBA servers, and
you still end up using all the distributed programming tricks you learned from
CORBA when you write EJBs. The big myth of EJB programming is that you don't
have to worry about scalability issues since the container does it for you.
The truth is that you still do because most containers blow.

3) EJB isn't quite as flexible as CORBA (see my EJB advantages later for the
downside to this). With CORBA you can code just about any type of servant
object you want, with EJB you are limited to whatever the container supports,
and only a few models with EJB itself (although, these are pretty flexible in
their own right).
4) EJB containers are still fairly new, and you could easily get locked into
something that evaporates a few years later. Many of your CORBA ORBs that are
available have been around for a long time and have a better track record.
5) Finally, there are more services available with CORBA 3.0,and they are
well defined. This is a debatable one though, since most vendors can't seem
to get these right.