There are lots of different aspects to a garden, and one that take a lot of time is organization! Organizing crop rotations, seed starting, harvesting times, companion planting, etc. - honestly the organization takes almost the longest. I started about 800 seeds inside of the 1500 plants that are in the garden now. However, it is really not that much work - a couple of hours a day probably, but this is how we "relax" around here.
Our garden is about 3400 sf plus about another 800 sf of raised beds. There are some plants that don't grow much the first year, such as asparagus and strawberries and other perennials, but after they are established these types of plants need very little maintenance and produce for years, which makes it well worth the wait. Also, if you consider the price of fresh fruit in the store like blueberries, raspberries, and grapes, the payoff even after the first year is tremendous. We planted blueberries last fall and already this spring we have a few quarts coming in. And I guarantee that at the end of the year, the pizza sauce we can with our tomatoes, peppers, garlic, and herbs is the best! There is no better flavor than food out of your own garden. My mom and I (with help here and there from the men) will can about 80 quarts in one day. It usually takes a good 12-15 hour day, but it is worth it for that sauce all year long.