I have a dual server setup, but only one does Alarm notification.
If I use Alarm Area Filtering via a workstation.startup file to hide alarms for area B, will that stop the notification system from sending alarms for the hidden area B?

No. Alarm area filtering only prevents alarms from being displayed on a particular workstation. It doesn't prevent the the tag from being loaded.

Let's say you have ServerA and ServerB with AreaA and AreaB. ServerA has no filtering so it will see alarms in both areas on the alarm page. ServerB has alarm area filtering set for AreaB. People logging into ServerB will only see alarms for AreaB on the alarm page. If there is an alarm in AreaA, the alarm will notify. Users logged into ServerB will see the alarm indicator flash and even receive a notification, but the alarm will not be displayed on the alarm page to be seen or acknowledged. If the user has access to the tag browser they will still have access to the tag properties.

If the idea is to prevent alarm notification from happening on ServerB, then set up an AlarmNotification server list with just ServerA on that list. Any alarm notification will just be made on from ServerA. Alarms can be acknowledged from either server but notified from just the one server. If it is offline, no notification will be made.