To get Git Bash, you actually install the "Git for Windows" package. Follow these steps to set it up correctly:
Choose between the or 64-bit standalone installer depending on your version of Windows. 1.5 Getting Started - Installing Git
The standard command-line interface for Unix-based systems. Git SCM: The full suite of Git version control tools.