1. Install command line tools

sudo xcode-select --install

2. Download and install MacPorts for your operating system version from https://www.macports.org/install.php.

3. Install VLC port.

sudo port install VLC

4. Find VLC port installation directory.

port contents vlc

For the rest of this guide, the installation directory is considered to be /Applications/MacPorts/VLC.app/Contents/MacOS.
If the location of your installation directory is different, use that for the following steps.

5. Create symbolic links to the development files in standard locations.

sudo ln -s /Applications/MacPorts/VLC.app/Contents/MacOS/include/vlc /usr/local/include/
sudo ln -s /Applications/MacPorts/VLC.app/Contents/MacOS/lib/* /usr/local/lib/

6. Set the VLC_PLUGIN_PATH environment variable.

export VLC_PLUGIN_PATH="/Applications/MacPorts/VLC.app/Contents/MacOS"

In order to permanently set the VLC_PLUGIN_PATH environment variable, add the command above to ~/.profile, ~/.bash_profile or ~/.bashrc. Please open a new terminal window in order for the change to take effect.

Follow the build instructions found at https://wiki.videolan.org/MacOSCompile.