Yes, you can embed videos in your website. You could upload the videos into your Media Library, but that is not the best choice. A much better alternative is to first upload your video to YouTube or Vimeo. Once you have uploaded your video to YouTube or Vimeo, copy the video URL. You can find the URL here:
Go back to your WordPress website and paste the video URL in the post or page editor and watch the magic. WordPress will automatically fetch the embed code and display your video for you. Publish or Update your changes.