Well it really depends. Apparent brightness is a combination of factors, but the most important one is exit pupil (the little circle of light in the ocular lens). You can calculate your exit pupil: Objective diameter divided by magnification=exit pupil). Notice how when you turn down the magnification on your scope in low light the image gets brighter? So a 3-9X50 would appear brighter than a 3-9X40 at higher powers. The catch is the image doesn't appear much brighter once the exit pupil is larger than 7mm, because 7mm is as large as most people's eyes can dialate.
The 40mm objective scopes are nice in scopes of moderate (say 3-10) magnifications because they provide adequate exit pupils for bright images, and they faciliate a low mounting height, and the subsequent excellent cheek weld. Proper cheek weld is extremely important for precision shooting.
Large objectives are particularly usefull on high-magnification scopes, because they are necessary to get useable exit pupils for low light performance at high magnifications. A 50mm scope at 10x provides a very nice 5mm exit pupil, bu the 40mm scope still provides a perfectly suitable 4mm exit pupil (similar to the 4.2mm exit pupil of your typical 10X42 binocular). Once you get to 15x, the larger objective becomes more beneficial as the 50mm scope now has a useable 3.3mm pupil, while the 40mm scope has a dim 2.6mm exit pupil.