There seems to be a bug whereby keep/tower/castle lords are scaling with the total number of nearby players, rather than just the number of attackers. This means that when 15 people attack a keep, and 25 defend, the lord will scale for 40 people. Add in iron guards and it's effectively scaled for 80. Such scaling is illogical and imbalanced and is presumably not intended. The range also appears to be very long, such that a zerg passing through the outer northeast gate of Bay keep will already scale up the lord.