22Deep Learning · Transfer Learning · UX · Read 11 min

Comparing two interfaces with transfer learning: adapting ResNet50 to UX perception

I adapted a pre-trained network to estimate UX dimensions and focused on a more useful question than exact score prediction: does the model preserve the human ranking between two webpages?

IN ONE SENTENCE

To compare two webpages, a useful model must first preserve the ordering of human ratings; low absolute error is insufficient when the ranking is wrong.

CONTEXT

DeepUX started from an applied research question: can a screenshot help compare two interfaces on aesthetics, usability or trustworthiness? I worked with the Web Design Prototypicality Dataset, which pairs webpage screenshots with human ratings across six perceptual dimensions.

SUMMARY

I first trained an AlexNet-inspired convolutional network from scratch, then adapted an ImageNet-pretrained ResNet50 to regression. The protocol separates feature extraction, training a new head and partial fine-tuning. Evaluation combines absolute error, R², Pearson correlation and Spearman rank correlation. For the intended use—choosing which of two webpages scores higher—correlation is more informative about preserved human ordering than MAE alone.

Key diagram

From an ImageNet backbone to pairwise comparison of two interfaces

01 · INPUTS

PAGE AInterface 01

PAGE BInterface 02

→SHARED WEIGHTS

02 · TRANSFER LEARNINGResNet50ImageNet · include_top=False

Frozen backbone → partial fine-tuning

→REGRESSION

03 · UX SCORES

A0.62

B0.41

Δ = +0.21 → A

04 · EVALUATE ORDERING

MAE

calibration

Pearson

shared variation

Spearman

relative order

Both screenshots pass through the same ResNet50 adapted to regression. Their scores are compared, while correlation evaluates whether the model preserves human ordering overall.

A research-based explainer grounded in the sources listed below. Some mechanisms are simplified without changing the overall conclusions.

Metric choice22

When comparing two interfaces, should we optimize the exact rating or their relative order?

I processed both screenshots through the same regression model, then measured whether predictions moved in the same direction as human judgments. MAE checks calibration; Pearson and Spearman reveal whether the model preserves the relative signal needed for an A/B comparison.

Next checks

Pearson and Spearman correlations

Pairwise accuracy by dimension

MAE, calibration and confidence intervals

Key idea

Transfer learning offers a strong starting point when UX data are limited, but architecture does not solve everything: human targets remain noisy, ImageNet differs from interfaces and fine-tuning can hurt generalization.

01 — The actual task

Comparing two webpages requires preserving an order, not merely predicting a rating.

DeepUX was not designed to claim that an interface is exactly 6.8 out of 10. Its original use takes screenshots A and B, selects a dimension—aesthetics, usability or trustworthiness—and indicates which page receives the higher model score. The useful output is therefore the sign of the difference between two predictions.

This changes the primary metric. Mean Absolute Error measures distance from human ratings. Pearson correlation measures whether values move together, while Spearman correlation focuses on whether their ordering is similar. A model may be poorly calibrated yet rank pages correctly—or achieve acceptable MAE by compressing scores while misordering examples.

02 — First architecture

A from-scratch CNN provides a baseline but must learn all of vision from limited data.

I first built an AlexNet-inspired network with three convolution, pooling and normalization blocks followed by two dense layers with dropout. A linear output turns image classification into regression over a continuous UX score. This architecture makes the entire pipeline controllable and exposes overfitting directly.

The preserved protocol includes about 3,145 valid screenshots, split into 2,516 training and 629 validation examples in an initial experiment. Data augmentation, early stopping and learning curves reduce some risks, but the network must still learn edges, composition, textures and their relationship with a noisy perceptual target at the same time.

03 — Transfer learning

ResNet50 contributes pretrained visual representations; a new head learns UX regression.

I loaded ResNet50 without its classification layer and retained weights learned on ImageNet. Global Average Pooling summarizes its activation maps; a dense head with dropout and a linear output then produces a continuous score. During the first phase, the backbone is frozen and only the new layers learn the mapping from visual representations to UX ratings.

This reduces the number of parameters to adapt and stabilizes learning on a small dataset. Yet the transfer moves knowledge from natural objects to interface screenshots. Some useful structures overlap—contrast, density, alignment and visual hierarchy—while the source domain does not contain many conventions specific to web design.

04 — Fine-tuning

Unfreezing the final blocks at a lower learning rate does not guarantee a better comparison.

After training the head, I unfroze the final ResNet50 layers and reduced the learning rate from 10⁻⁴ to 10⁻⁵. The aim was to adapt high-level representations to interface-specific forms without abruptly erasing pretrained features.

The saved traces show why this phase must be evaluated rather than assumed beneficial. With the frozen backbone, the best validation MAE in this run falls near 0.53. It rises at the start of fine-tuning before decreasing again, and the run stops before full convergence. More trainable layers add capacity as well as overfitting and instability risk.

05 — Evaluation

MAE, R² and correlation answer different questions.

I tracked MAE for calibration, R² for explained variance, and Pearson and Spearman correlations for the relative structure of predictions. The experiments also use folds and a baseline predicting the training-set mean. The model should beat that reference, but above all produce coherent differences between unseen webpages.

In one preserved trustworthiness run, aggregated predictions after the second stage reach a Pearson correlation of approximately 0.26 between target and prediction. The signal is positive but modest: it supports a comparison prototype, not an autonomous judgment of UX quality. Explicit pairwise accuracy—the proportion of pairs whose order is predicted correctly, with confidence intervals—would be the most direct product metric to add.

06 — What I learned

The model learns a relative visual signal; it does not measure lived experience.

A screenshot contains cues associated with first impressions: density, balance, contrast, organization and graphic conventions. It contains neither interaction fluency nor understanding after use or task success. DeepUX should therefore shortlist or compare variants before expert review and user research.

The next protocol would split websites by domain, explicitly compare the from-scratch CNN, frozen backbone and several fine-tuning depths, then compute pairwise accuracy for each UX dimension. I would also add confidence intervals, output calibration and a qualitative analysis of cases where model and human ratings diverge.

Primary source

DeepUX research notebooks and prototype

A synthesis of my experiments on the Web Design Prototypicality Dataset: a from-scratch CNN, pretrained ResNet50, frozen-backbone training, partial fine-tuning and error- and correlation-based evaluation.

Selected references

He et al. (2016) — Deep Residual Learning for Image Recognition
Miniukovich & Figl (2024) — Web Design Prototypicality Dataset
ImageNet pre-training
Pearson, Spearman, MAE and R²