Ok, lets start making it clear who the initiator (SRC) and receiver (DST) are. SRC is the person/computer who wants to talk to you and makes the first attempt to do so. Receiver is the person who will either respond to the attempt made by the SRC or just ignore it.
Now, in ISA, please remember that outgoing IPs are ALWAYS the first external IP of the NIC (if you perform NAT from source internal to external). This is true only in a scenario where ISA is the final hop to reach the internet.
ISA manages outgoing requests through PAT (port address translation) but when it comes to incoming requests such as a published webserver etc, ISA can be reached on any external IPs which you specify in the Wizard.
So in short, if the SRC is internal and the DST is external, ISA will use it's first external IP address and, if the SRC is external and the DST is internal, and if you have a corresponding rule/listener, ISA will accept incoming connections using that IP you specify in the wizard.
This is especially important if you performing reverse dns settings esp for SMTP MX servers. Always to remember to register your ISA's first external IP along with your actual SMTP IP as your reverse DNS settings. Otherwise, your org's email can identified as a potential SPAMMER by reverse lookup checks done by SMTP engines.