Estimating development time is hard because:
- Software development is a creative process. You get better at estimating similar projects and components but creating something new requires some creative innovation.
- Overconfidence. You can train yourself to avoid overconfidence. But asking everyone on the team to do the same is hard. Programmers frequently take more than they can chew.
- You cannot predict the future. There will be obstacles that you didn't consider when estimating. If you consider every single possibility and write the full specifications, you end up wasting more time than you would have by just writing the program.
Giving reasonable estimates takes practice. Never give on-the-spot estimates on your tasks, take your time to plan the outline so you can really train estimating. You will get better at it. If you aren't allowed the time to prototype or investigate carefully, say that the following estimate isn't going to be really good.
An estimate is never a commitment. You can't commit to something you're not 100% sure.
Making accurate long-term estimates is fundamentally impossible. If you later realize you made estimation error, inform about it as soon as possible. It isn't rare to estimate something big to take 3 weeks and it finally takes 3 months. Give your best guess but make sure people understand it's a guess.
Dividing is the best cure for bad estimates. The best countermeasure for inaccurate estimates is shortening the development cycle. Try using rapid, one week sprints. Split tasks to tickets take fixed time of 0, 2, 4 or 8 hours. No more. These short sprints place hard limit to horrifically bad estimates as you cannot estimate something to take two weeks, you will split functionalities to smaller pieces.