Project 2: DoppelGanger - Find Celebrity Look-Alike

 

1) Overview (What this code is doing)

This code implements a face recognition workflow using Dlib’s face recognition model. The key idea is broken into two segments:

  • Enrollment: for each known celebrity image, compute a 128-dimensional embedding and store it with the person’s label.
  • Testing: for each unknown/test image, compute its 128D embedding and find the closest enrolled embedding using a minimum distance rule. The closest match is shown as the “look-alike”.

The deep network is trained once on a large dataset, and the application enrolls a small number of images per identity to build a searchable database of embeddings.

2) Imports and Matplotlib configuration

python
import os,random,glob
import dlib
import cv2
import numpy as np
import matplotlib.pyplot as plt
import time
%matplotlib inline

Explanation

  • os/random/glob: filesystem traversal, picking random folders, and finding test images.
  • dlib: provides:
    • frontal face detector
    • facial landmark predictor
    • face recognition network
  • cv2 (OpenCV): used for loading images and color conversion (BGR↔RGB).
  • numpy: store embeddings and compute distances efficiently.
  • matplotlib: display images inline.

Plotting defaults

import matplotlib
matplotlib.rcParams['figure.figsize'] = (6.0,6.0)
matplotlib.rcParams['image.cmap'] = 'gray'
matplotlib.rcParams['image.interpolation'] = 'bilinear'

This just sets figure size and display style; it doesn’t affect recognition.

Load models

PREDICTOR_PATH = '../resource/lib/publicdata/models/shape_predictor_68_face_landmarks.dat'
FACE_RECOGNITION_MODEL_PATH = '../resource/lib/publicdata/models/dlib_face_recognition_resnet_model_v1.dat'

faceDetector = dlib.get_frontal_face_detector()
shapePredictor = dlib.shape_predictor(PREDICTOR_PATH)
faceRecognizer = dlib.face_recognition_model_v1(FACE_RECOGNITION_MODEL_PATH)

Explanation

You create three critical components:

  1. Face detector (get_frontal_face_detector)
    Finds face bounding boxes in the image.

  2. Facial landmark detector (shape_predictor)
    Produces 68 landmark points from the image. This is commonly used to normalize/alignment and to guide descriptor extraction.

  3. Face recognition model (face_recognition_model_v1)
    Produces an embedding for a detected face. Enrolling a person means passing a few images through the network and saving these results. Images of the same person should be close, while different people should be far apart.

Explore dataset (folders = labels, mapping = names)

faceDatasetFolder = '../resource/asnlib/publicdata/celeb_mini'
labelMap = np.load("../resource/asnlib/publicdata/celeb_mapping.npy", allow_pickle=True).item()
labelMap

subfolders = os.listdir(faceDatasetFolder)
random_folder = random.choice(subfolders)

celebname = labelMap[random_folder]
imagefiles = os.listdir(os.path.join(faceDatasetFolder, random_folder))

for file in imagefiles:
    fullPath = os.path.join(faceDatasetFolder,random_folder,file)
    im = cv2.imread(fullPath)
    plt.imshow(im[:,:,::-1])
    plt.show()
    print("File path = {}".format(fullPath))
    print("Celeb Name: {}".format(celebname))

Explanation

  • The dataset is organized such that each subfolder corresponds to one identity.
  • labelMap translates a folder-id into the celebrity’s actual name.
  • OpenCV loads images in BGR; im[:,:,::-1] flips channels to RGB for Matplotlib visualization.

Enrollment (build the embedding database)

Instead of training a recognizer on your small dataset, this uses a pre-trained deep model and enrolls each identity by computing and storing their descriptors. These descriptors act like coordinates in a high-dimensional space where same-identity faces cluster together.

enrolled_embeddings = []
enrolled_labels = []
enrolled_imgpaths = []

subfolders = sorted(os.listdir(faceDatasetFolder))

for label in subfolders:
    folderPath = os.path.join(faceDatasetFolder, label)
    if not os.path.isdir(folderPath):
        continue

    imageFiles = sorted(os.listdir(folderPath))

    for file in imageFiles:
        imgPath = os.path.join(folderPath, file)
        im = cv2.imread(imgPath)
        if im is None:
            continue

        # Convert BGR -> RGB because dlib uses RGB by default
        imDlib = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)

        # Detect faces
        faces = faceDetector(imDlib, 1)
        if len(faces) == 0:
            continue

        # Use the largest face if multiple are found
        face = max(faces, key=lambda r: r.width() * r.height())

        # Landmarks
        shape = shapePredictor(imDlib, face)

        # 128D embedding (face descriptor)
        faceDescriptor = faceRecognizer.compute_face_descriptor(imDlib, shape)
        faceDescriptor = np.array(faceDescriptor, dtype=np.float32)

        enrolled_embeddings.append(faceDescriptor)
        enrolled_labels.append(label)
        enrolled_imgpaths.append(imgPath)

enrolled_embeddings = np.vstack(enrolled_embeddings)  # shape: (N, 128)
enrolled_labels = np.array(enrolled_labels)
enrolled_imgpaths = np.array(enrolled_imgpaths)

print("Enrollment complete!")
print("Total enrolled face images:", enrolled_embeddings.shape[0])
print("Embedding dimension:", enrolled_embeddings.shape[1])

Step-by-step explanation

  1. Create storage lists

    • enrolled_embeddings: one vector per image (eventually shape (N,128)).
    • enrolled_labels: the identity label (folder id).
    • enrolled_imgpaths: keep file paths so it can display a representative “best match” image later.

  2. Loop over each identity folder
    Each folder corresponds to a celeb label (like n00002023).

  3. Read each image

    • cv2.imread returns BGR.
    • If im is None, the file couldn’t be read and is skipped.

  4. Convert BGR → RGB
    Dlib expects RGB images by default, so the conversion is essential for correctness/consistency. The OpenCV course material explicitly emphasizes that model expectations must be respected when feeding images to detectors and networks.

  5. Detect faces

    • faces = faceDetector(imDlib, 1) returns rectangles.
    • If none found, skip.

  6. If multiple faces, pick the largest
    This is a practical heuristic: in celebrity photos, the main subject is often largest.

  7. Compute landmarks
    shapePredictor(imDlib, face) finds the 68 landmarks.

  8. Compute the 128D embedding
    compute_face_descriptor(imDlib, shape) outputs the face descriptor.
    Per the OpenCV course material, this 128D descriptor is the “feature” used in the high-dimensional embedding space for matching.

  9. Stack results
    np.vstack turns the Python list into an (N,128) matrix, enabling fast vectorized distance computations.

Testing (find the closest celebrity by minimum distance)

matplotlib.rcParams['figure.figsize'] = (12.0,12.0)

testImages = glob.glob('../resource/asnlib/publicdata/test-images/*.jpg')
print("Found test images:", testImages)

for test in testImages:
    im = cv2.imread(test)
    imDlib = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)

    faces = faceDetector(imDlib, 1)

    celeb_name = "Unknown"
    celeb_img_rgb = np.zeros((200, 200, 3), dtype=np.uint8)

    if len(faces) == 0:
        celeb_name = "No face found"
    else:
        face = max(faces, key=lambda r: r.width() * r.height())

        shape = shapePredictor(imDlib, face)

        testDescriptor = faceRecognizer.compute_face_descriptor(imDlib, shape)
        testDescriptor = np.array(testDescriptor, dtype=np.float32)  # (128,)

        diffs = enrolled_embeddings - testDescriptor.reshape(1, -1)
        dists = np.sqrt(np.sum(diffs * diffs, axis=1))  # Euclidean (L2) distance

        bestIdx = np.argmin(dists)
        bestLabel = enrolled_labels[bestIdx]

        celeb_name = labelMap[bestLabel]

        best_img_path = enrolled_imgpaths[bestIdx]
        celeb_img_bgr = cv2.imread(best_img_path)
        if celeb_img_bgr is not None:
            celeb_img_rgb = cv2.cvtColor(celeb_img_bgr, cv2.COLOR_BGR2RGB)

    plt.subplot(121)
    plt.imshow(imDlib)
    plt.title("test img")

    plt.subplot(122)
    plt.imshow(celeb_img_rgb)
    plt.title("Celeb Look-Alike={}".format(celeb_name))
    plt.show()

Explanation (what happens for each test image)

  1. Load test image + convert to RGB.
  2. Detect face; if none found, report “No face found”.
  3. Compute landmarks and then compute the 128D embedding for the test face.
  4. Compare test embedding to all enrolled embeddings
    • diffs = enrolled_embeddings - testDescriptor
    • dists = L2 distance for each enrolled image
  5. Pick minimum distance (argmin) as the closest match (nearest neighbor).
  6. Map label → celebrity name using labelMap.
  7. Display
    • Left: test image
    • Right: the enrolled image that produced the closest embedding.

Comments

Post a Comment