Capstone Project Choice: Social Distance Alert

I chose to create my own project from scratch. The project implements a solution for determining if a crowded area is in violation of the 6-foot rule for social distancing. The input is a directory containing photos of groups of people in jpeg format, the tool looks for faces in the photos, determines relative distances among them, and compares the distances to the 6-foot rule for social distancing.

The actual algorithm that determines relative distances and flags violators is a prototype only. It is not intended to provide accurate results to any extent. The algorithm is simply used as backbone and motivation for the classes that handle all preparatory steps before the algorithm can run, and all concluding tasks that deploy the end result.

---

Expected Behavior

The output should be a log file named SocialDistanceAlert.log and an output directory containing copies of each original jpeg file provided as input. The log file should contain a message for each input file indicating how many violators were detected and the total number of people in the group. The image files in the output directory should each be named after the original image file, contain green squares around the faces of all individuals in the group, and red squares around the faces of individuals who violate the social distancing rule. Note that the output directory is provided as a command line argument and created if not already existent. Also, currently, the only file format supported is JPEG files with extension jpeg or jpg. Below is a sample output image showing that out of 8 individuals, 6 were detected by the algorithm, and 3 were deemed to be closer to each other than 6 feet. The focus of the project is to exercise the concepts and lessons found in the nanodegree program. It is not a goal to provide a production ready detection algorithm and to analyse or fine-tune accuracy.

---

Project Structure and Design

The project is composed by a library called SocialDistanceAlert which provides all functionality for the project, and a main entry point which makes use of the libray linking to it statically. The following table shows what the subsystems within the library are, and how they interface:

Subsystem name	Instances	Functionality	Interfaces Exported	Interfaces Consumed
CommandLineParser	1	Parse CLI arguments; Provide CLI help; First pass validation;	parseCommandLine moveDirectoryPaths	None
InputOutputHandler	1	Full input data validation; Load images to memory;	validateAllFilenames processFiles getAllLoadedImages	LOG_INFO LOG_ERROR
Logger	1	Asynchronously writes messages coming from many threads to a single file;	LOG_INFO LOG_ERROR startWriter stopWriter	None
HaarCascadeDetector	1 per input file	Detects faces in images and determines faces that are too close;	addModel prepare detect finalize deploy	LOG_INFO LOG_ERROR

A rough graphical schematics of the above interfaces is shown in the following figure:

The following table describes all subsystems in greater detail, including the specific classes that were written for each subsystem, including any relevant data structures, algorithms, and desing patterns used:

Subsystem name	Classes	Data Structures	Algorithms	Patterns
CommandLineParser	CommandLineParser class	1 hash table of directories keyed by type.	Parses and validates the arguments provided by user.	None
InputOutputHandler	InputOutputHandler class	1 output directory, a hash table of input directories keyed by file extension, 1 hash table of loaded images keyed by filename.	Validates individual filenames and loads all ifle to memeory.	Meyer’s Singleton
Logger	Logger class	1 queue of messages to log and 1 thread to write to disk.	Accepts messages containing a std::ostringstream and queues them to be saved to disk asynchronously.	Meyer’s Singleton
HaarCascadeDetector	HaarCascadeDetector class	1 cv::Mat to store image, at least 1 cv::CascadeClassifier, a vector of cv::Rect for detected heads, and a vector of cv::Rect for offenders.	Detects heads in an image, estimates if rectangles delimiting detections are too close in 3D.	Message Queue

The threading model chosen intends to provide a buffer between multiple worker threads that perform the heavy-lifting tasks of the project and a single thread solely responsible for successfully writing them to disk. The following picture details the model and approximately depicts the level of indirection introduced by the message queue:

The calls to MessageQueue<T>::receive() are made much for often than shown, and the final call is accompanied by a flush procedure. Those details are not shown above for simplicity.

---

How To Build This Project

This project is intended to target the Udacity VM environment. That environment is based on Ubuntu 16.04, and has a few installed development packages, including make 4.1, cmake 3.13, and g++ 5.5.

The minimum required versions for this project are: make 4.1, cmake 3.10 and g++ 9. The version for the gcc compiler needs to be at least 9 so to include support for std::filesystem library.

In order to accommodate for that in the Udacity VM environment, run the following:

sudo apt update --fix-missing
sudo apt install gcc-9 g++-9
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 100 --slave /usr/bin/g++ g++ /usr/bin/g++-9

Verify that gcc in use now is version 9:

gcc --version

Should print something like:

gcc (Ubuntu 9.4.0-1ubuntu1~16.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Other than that, this project should build in any recent Ubuntu linux environment. It does depend on OpenCV. This dependency can typically be provided by running:

sudo apt install libopencv-dev

A full list of install instructions can be found at OpenCV. After that dependency is satisfied, the project can be build by using any modern build system and compiler toolchain. I chose to use cmake and make for build system, and this project comes with all the cmake files needed to build it in a typical Ubuntu Linux environment. It has been tested using make, cmake and g++. The suggested steps are as follows:

$> cd <project_path>
$> cmake -B make-build-debug -DCMAKE_BUILD_TYPE=Debug -G "Unix Makefiles"
$> cmake --build make-build-debug -j4

---

How to Install This Project

It is important to perform a install as proper relative placement of files used during runtime is only guaranteed within the installation directory. If no change is made during install, the tool is installed in the directory named local (inside build, if the suggestions above are followed).

$> cmake --install make-build-debug

---

How to Run This Project

Once built and installed as suggested above, in the cmake build directory, simply do:

$> cd local/bin
$> ./SocialDistanceAlertSystem --jpeg=input --outdir=output

That should print out something like this:

Input path provided input points to a directory as expected.
Files with extension .JPG found:
* input/crowd_A.jpg
* input/crowd_C.jpg
* input/crowd_D.jpg
* input/crowd_B.jpg
Detected 7 offenders in a group of 7 individuals in file crowd_D.jpg. Please look for respective output file.
Detected 0 offenders in a group of 2 individuals in file crowd_B.jpg. Please look for respective output file.
Detected 3 offenders in a group of 6 individuals in file crowd_C.jpg. Please look for respective output file.
Detected 6 offenders in a group of 11 individuals in file crowd_A.jpg. Please look for respective output file.
Processed 4 images from input directory in 4 threads.
Stopping logger now. There will be only one more message written to log file after this.
This is the last message written to log file.
Writer done!
Writer stopped!

All needed files are located in the directory named local within the cmake build directory. If you use the above suggested command sequence, then the installation full path will be in <project root>/build/local. Other than that, just provide the directory path containing the pictures you want to have processed.

For added convenience reviewing the output, install and use fim to review the images:

cat SocialDistanceAlert.log
sudo apt install fim
fim output/*.jpg

---

Rubric Points Addressed

1) README (All Rubric Points REQUIRED):

• A README with instructions is included with the project:
  • The README is included with the project and has instructions for building and running the project. Addressed in How to Run This Project.
  • If any additional libraries are needed to run the project, these are indicated with cross-platform installation instructions. Addressed in How to Install This Project.
  • You can submit your writeup as markdown or pdf. Addressed by this very file, markdown was chosen as the only format because of the hyperlinking functionality.
• The README indicates which project is chosen:
  • The README describes the project you have built. Addressed in project introduction.
  • The README also indicates the file and class structure (addressed in Project Structure), along with the expected behavior or output of the program (addressed in Expected Behavior).
• The README includes information about each rubric point addressed.
  • The README indicates which rubric points are addressed (you are reading it). The README also indicates where in the code (i.e. files and line numbers) that the rubric points are addressed (see the respective sections below).

2) Compiling and Testing (All Rubric Points REQUIRED):

• The submission must compile and run.
  • The project code must compile and run without errors. The project was verified to build and run without errors within the Udacity VM environment.
  • We strongly recommend using cmake and make, as provided in the starter repos. If you choose another build system, the code must compile on any reviewer platform. I chose cmake and make as in the starter repo, addressed in How to Build This Project.

3) Loops, Functions, I/O:

• The project demonstrates an understanding of C++ functions and control structures:
  • A variety of control structures are used in the project. Sample from [InputOutputHandler.cpp line 87] :

    LOG_INFO("Creating %s: %s", common::outputDirectoryKey, directory.c_str());
    if (std::filesystem::create_directory(directory))
    {
       LOG_INFO("Output directory successfully created.");
    }
    else
    {
      LOG_ERROR("Directory creation failed: %s", directory.c_str());
    }
    

  • The project code is clearly organized into functions. This can be seen all across project, for example see [CommandLineParser.cpp line 21] .
• The project reads data from a file and processes the data, and the program writes data to a file:
  • The project reads data from an external file. Sample from [InputOutputHandler.cpp line 133] :

    m_loadedImages[filename] = cv::imread(imagePath);
    

  • The project writes data to a file as part of the necessary operation of the program. Sample from [HaarCascadeDetector.cpp line 159] :

    void HaarCascadeDetector::deploy()
    {
      cv::imwrite(InputOutputHandler::instance().getOutputDirectoryPath(), m_image);
    }
    

• The project accepts user input and processes the input:
  • The project accepts input from a user as part of the necessary operation of the program. See the [CommandLineParser.cpp line 21] :

    void CommandLineParser::parseCommandLine(int argc, char* argv[])
    {
      cv::CommandLineParser parser(argc, argv, common::PARSING_KEYS);
      auto jpegInputDirectoryPath = parser.get<std::string>("jpeg");
      auto outputDirectoryPath = parser.get<std::string>("outdir");
    
      parser.about(common::description);
      if (parser.has("help") ||
          jpegInputDirectoryPath.empty() ||
          outputDirectoryPath.empty())
      {
          parser.printMessage();
          return;
      }
      m_directoryPaths->emplace(common::jpegExtention, jpegInputDirectoryPath);
      m_directoryPaths->emplace(common::outputDirectoryKey, outputDirectoryPath);
    }
    

Object-Oriented Programming

• The project uses Object-Oriented Programming techniques: The project code is organized into classes with class attributes to hold the data, and class methods to perform tasks. All across project, see Project Structure.
• Classes use appropriate access specifiers for class members: All class data members are explicitly specified as public, protected, or private. All across project, for example see [InputOutputHandler.h line 13] .
• Class constructors utilize member initialization lists: All class members that are set to argument values are initialized through member initialization lists. All across project where appropriate, for example see constructor of [InputOutputHandler.cpp line 17] .
• Classes abstract implementation details from their interfaces: All class member functions document their effects, either through function names, comments, or formal documentation. Member functions do not change program state in undocumented ways. All across project, for example see [CommandLineParser.cpp line 21] .
• Classes encapsulate behavior: Appropriate data and functions are grouped into classes. Member data that is subject to an invariant is hidden from the user. State is accessed via member functions. All across project, for example see [HaarCascadeDetector.cpp line 9] .
• Classes follow an appropriate inheritance hierarchy: Inheritance hierarchies are logical. Composition is used instead of inheritance when appropriate. Abstract classes are composed of pure virtual functions. Override functions are specified. Composition is used in Logger as it is composed by a MessageQueue as in [Logger.h line 70] .
• Overloaded functions allow the same function to operate on different parameters: One function is overloaded with different signatures for the same function name. . All across project.
• Derived class functions override virtual base class functions: One member function in an inherited class overrides a virtual base class member function. . All across project.
• Templates generalize functions in the project: One function is declared with a template that allows it to accept a generic parameter. MessageQueue is a template class, and its member function void send(T&& msg) accepts a generic parameter [Logger.h line 61] .

Memory Management

• The project makes use of references in function declarations: At least two variables are defined as references, or two functions use pass-by-reference in the project code. The member functions [void Logger::log(const std::string& label, const char * format, va_list args) in Logger.h line 80] and [void HaarCascadeDetector::prepare(const std::string& filename, cv::Mat& image) in HaarsCascadeDetector.h line 36] use pass-by-reference.
• The project uses destructors appropriately: At least one class that uses unmanaged dynamically allocated memory, along with any class that otherwise needs to modify state upon the termination of an object, uses a destructor. [Logger uses a destructor as in Logger.cpp line 52] to make sure an output file stream is closed.
• The project uses scope / Resource Acquisition Is Initialization (RAII) where appropriate: The project follows the Resource Acquisition Is Initialization pattern where appropriate, by allocating objects at compile-time, initializing objects when they are declared, and utilizing scope to ensure their automatic destruction. All across project, see [main() line 14] for scoped execution flow.
• The project follows the Rule of 5: For all classes, if any one of the copy constructor, copy assignment operator, move constructor, move assignment operator, and destructor are defined, then all of these functions are defined. [The Logger defines/deletes all as appropriate as in Logger.h line 20] .
• The project uses move semantics to move data, instead of copying it, where possible: For classes with move constructors, the project returns objects of that class by value, and relies on the move constructor, instead of copying the object. The CommandLineParser moves the parsed directory paths out to clients in [std::unique_ptr<std::unordered_map<std::string, std::string» CommandLineParser::moveDirectoryPaths() as in CommandLineParser.cpp line 45] .
• The project uses smart pointers instead of raw pointers: The project uses at least one smart pointer: unique_ptr, shared_ptr, or weak_ptr. The project does not use raw pointers. The project does not directly call new or delete, does not allocate memory and handles it by direct use of raw pointers, and uses smart pointers in several locations, for example see [InputOutputHandler.h line 15] .

Concurrency

• The project uses multithreading: The project uses multiple threads in the execution. Please see [main() line 51] , or in a short form without error handling:

    std::vector<std::future<int>> tasks;
    for (const auto& [filename, matImage]: *allImages)
    {
        tasks.emplace_back(std::async(worker, std::ref(filename), std::ref(matImage)));
    }
    for(auto& ftr : tasks) ftr.get();

• A promise and future is used in the project: A promise and future is used to pass data from a worker thread to a parent thread in the project code. The member functions void Logger::messageWriter(std::future<void> stopWriterFuture) and void Logger::stopWriter() use future/promise to communicate a request to stop the worker thread that writes the log to disk as in [Logger.cpp line 14] .

void Logger::messageWriter(std::future<void> stopWriterFuture)
{
    while (stopWriterFuture.wait_for(std::chrono::milliseconds(10)) != std::future_status::ready)
    {
        auto msg = m_messages.receive();
        m_theLogFile << msg.str();
    }
}

• A mutex or lock is used in the project: A mutex or lock (e.g. std::lock_guard or std::unique_lock) is used to protect data that is shared across multiple threads in the project code. The Logger uses a std::mutex primarily to protect a MessageQueue<std::ostringstream> and a std::ofstream which are shared across threads [see Logger.h line 38] .
• A condition variable is used in the project: A std::condition_variable is used in the project code to synchronize thread execution. A std::condition_variable is used in the MessageQueue to synchronize execution of a threads sending messages to be written to disk and the thread that writes messages to disk [see Logger.h line 53] . —