BACnet describes devices and their points. Modbus only moves numbers. Most sites need both, read into one model.
They solve different problems
BACnet was designed for building automation. Devices describe themselves as objects with properties, such as an analog input with a present value and units. Modbus was designed to move register values between a controller and a device. It has no concept of a point name or a unit.
Discovery and self-description
A BACnet network can be searched with Who-Is, and each device can list its objects. A Modbus device cannot be asked what it contains. You need the manufacturer register map to know that holding register 40012 is a voltage.
Networks and speed
BACnet/IP runs over Ethernet and routes between networks, while BACnet MS/TP runs on shared RS-485 trunks. Modbus RTU also uses RS-485, and Modbus TCP uses Ethernet on port 502. Serial trunks in both cases are slow and shared, so polling must be planned.
Where each one fits
Use BACnet for HVAC plant, VAV boxes and building controllers. Use Modbus for energy meters, variable frequency drives, PLCs and a great deal of older equipment. A commercial building commonly has both.
Reading both together
The practical problem is not choosing one, but reading both. A normalized tag model gives a BACnet temperature and a Modbus meter reading the same shape, with a name, unit, timestamp and quality, so they can be trended and alerted on together.