Skip to content

Bidder Configuration Fixes & URL Validation #4126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Bidder Configuration Fixes & URL Validation #4126

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
704ab67
Bidder Configuration Fixes
AntoxaAntoxic Aug 8, 2025
c6c7305
Upgrade URL Validation
AntoxaAntoxic Sep 4, 2025
704b179
Merge remote-tracking branch 'origin/master' into bidder-configs-fixes
AntoxaAntoxic Sep 4, 2025
a756d98
Upgrade URL Validation
AntoxaAntoxic Sep 4, 2025
8a27c4e
Merge remote-tracking branch 'origin/master' into bidder-configs-fixes
AntoxaAntoxic Sep 19, 2025
e1d2067
Make URL encoding RFC 3986 compliant
AntoxaAntoxic Sep 26, 2025
a3036fc
Merge remote-tracking branch 'origin/master' into bidder-configs-fixes
AntoxaAntoxic Oct 7, 2025
e5b02da
Small bidder updates
AntoxaAntoxic Oct 7, 2025
8743663
Fix pom.xml
AntoxaAntoxic Oct 7, 2025
c0446ec
Fix pom.xml
AntoxaAntoxic Oct 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...i/channel/identity/v1/hooks/LiveIntentOmniChannelIdentityProcessedAuctionRequestHook.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ public LiveIntentOmniChannelIdentityProcessedAuctionRequestHook(LiveIntentOmniCh
double logSamplingRate) {

this.config = Objects.requireNonNull(config);
HttpUtil.validateUrlSyntax(config.getIdentityResolutionEndpoint());
HttpUtil.validateUrl(config.getIdentityResolutionEndpoint());
this.mapper = Objects.requireNonNull(mapper);
this.httpClient = Objects.requireNonNull(httpClient);
this.logSamplingRate = logSamplingRate;
Expand Down
3 changes: 2 additions & 1 deletion ...server/hooks/modules/com/scientiamobile/wurfl/devicedetection/model/WURFLEngineUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import com.scientiamobile.wurfl.core.WURFLEngine;
import com.scientiamobile.wurfl.core.cache.LRUMapCacheProvider;
import com.scientiamobile.wurfl.core.cache.NullCacheProvider;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.hooks.modules.com.scientiamobile.wurfl.devicedetection.config.WURFLDeviceDetectionConfigProperties;
import org.prebid.server.hooks.modules.com.scientiamobile.wurfl.devicedetection.exc.WURFLDeviceDetectionException;

Expand Down Expand Up @@ -56,7 +57,7 @@ private static String wurflFilePathFromConfig(WURFLDeviceDetectionConfigProperti

public static String extractWURFLFileName(String wurflSnapshotUrl) {
try {
final URI uri = new URI(wurflSnapshotUrl);
final URI uri = new URIBuilder(wurflSnapshotUrl).build();
final String path = uri.getPath();
return path.substring(path.lastIndexOf('/') + 1);
} catch (Exception e) {
Expand Down
19 changes: 0 additions & 19 deletions extra/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,7 @@
<commons.collections.version>4.4</commons.collections.version>
<commons.compress.version>1.27.1</commons.compress.version>
<commons-math3.version>3.6.1</commons-math3.version>
<commons-validator.version>1.10.0</commons-validator.version>
<scram.version>2.1</scram.version>
<httpclient.version>4.5.14</httpclient.version>
<ipaddress.version>5.5.1</ipaddress.version>
<oshi.version>6.8.0</oshi.version>
<json-schema-validator.version>1.5.6</json-schema-validator.version>
Expand Down Expand Up @@ -136,23 +134,6 @@
<artifactId>commons-math3</artifactId>
<version>${commons-math3.version}</version>
</dependency>
<dependency>
<groupId>commons-validator</groupId>
<artifactId>commons-validator</artifactId>
<version>${commons-validator.version}</version>
</dependency>
<!-- TODO: refactor code to replace URIBuilder with something else so that this dep can be removed -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>${httpclient.version}</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.github.seancfoley</groupId>
<artifactId>ipaddress</artifactId>
Expand Down
8 changes: 2 additions & 6 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,12 +111,8 @@
<artifactId>commons-math3</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<dependency>
<groupId>commons-validator</groupId>
<artifactId>commons-validator</artifactId>
<groupId>org.apache.httpcomponents.client5</groupId>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this dependency. The whole point of this task was to get rid of apache HTTP client which creates dependency issues. This entire dependency is pulled in to use a URL validator and some utils. Which should be available separately.

<artifactId>httpclient5</artifactId>
</dependency>
<dependency>
<groupId>com.github.seancfoley</groupId>
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/prebid/server/auction/IpAddressHelper.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import inet.ipaddr.IPAddressString;
import inet.ipaddr.IPAddressStringParameters;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.conn.util.InetAddressUtils;
import org.apache.hc.core5.net.InetAddressUtils;
import org.prebid.server.auction.model.IpAddress;
import org.prebid.server.log.Logger;
import org.prebid.server.log.LoggerFactory;
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/prebid/server/bidder/adgeneration/AdgenerationBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
import io.vertx.core.http.HttpMethod;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.client.utils.URIBuilder;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.bidder.Bidder;
import org.prebid.server.bidder.adgeneration.model.AdgenerationResponse;
import org.prebid.server.bidder.model.BidderBid;
Expand Down
13 changes: 7 additions & 6 deletions src/main/java/org/prebid/server/bidder/adkernel/AdkernelBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ public class AdkernelBidder implements Bidder<BidRequest> {
new TypeReference<>() {
};

private static final String ZONE_ID_MACRO = "{{ZoneId}}";

private static final String MF_SUFFIX = "__mf";
private static final String MF_SUFFIX_BANNER = "b" + MF_SUFFIX;
private static final String MF_SUFFIX_VIDEO = "v" + MF_SUFFIX;
Expand All @@ -49,11 +51,11 @@ public class AdkernelBidder implements Bidder<BidRequest> {

private static final int MF_SUFFIX_LENGTH = MF_SUFFIX.length() + 1;

private final String endpointTemplate;
private final String endpointUrl;
private final JacksonMapper mapper;

public AdkernelBidder(String endpointTemplate, JacksonMapper mapper) {
this.endpointTemplate = HttpUtil.validateUrl(Objects.requireNonNull(endpointTemplate));
public AdkernelBidder(String endpointUrl, JacksonMapper mapper) {
this.endpointUrl = HttpUtil.validateUrl(Objects.requireNonNull(endpointUrl));
this.mapper = Objects.requireNonNull(mapper);
}

Expand Down Expand Up @@ -183,10 +185,9 @@ private HttpRequest<BidRequest> createHttpRequest(Map.Entry<ExtImpAdkernel, List
App app) {

final ExtImpAdkernel impExt = extAndImp.getKey();
final String uri = endpointTemplate.formatted(impExt.getZoneId());
final String uri = endpointUrl.replace(ZONE_ID_MACRO, HttpUtil.encodeUrl(impExt.getZoneId().toString()));

final MultiMap headers = HttpUtil.headers()
.add(HttpUtil.X_OPENRTB_VERSION_HEADER, "2.5");
final MultiMap headers = HttpUtil.headers().add(HttpUtil.X_OPENRTB_VERSION_HEADER, "2.5");

final BidRequest outgoingRequest = createBidRequest(extAndImp.getValue(), requestBuilder, site, app);

Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/prebid/server/bidder/adnuntius/AdnuntiusBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
import org.apache.commons.lang3.BooleanUtils;
import org.apache.commons.lang3.ObjectUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.client.utils.URIBuilder;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.bidder.Bidder;
import org.prebid.server.bidder.adnuntius.model.request.AdnuntiusMetaData;
import org.prebid.server.bidder.adnuntius.model.request.AdnuntiusNativeRequest;
Expand Down
11 changes: 4 additions & 7 deletions src/main/java/org/prebid/server/bidder/adocean/AdoceanBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,9 @@
import org.apache.commons.collections4.MapUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.math.NumberUtils;
import org.apache.http.NameValuePair;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.client.utils.URLEncodedUtils;
import org.apache.http.message.BasicNameValuePair;
import org.apache.hc.core5.http.NameValuePair;
import org.apache.hc.core5.http.message.BasicNameValuePair;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.bidder.Bidder;
import org.prebid.server.bidder.adocean.model.AdoceanResponseAdUnit;
import org.prebid.server.bidder.model.BidderBid;
Expand All @@ -37,9 +36,7 @@

import java.io.IOException;
import java.math.BigDecimal;
import java.net.URI;
import java.net.URISyntaxException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
Expand Down Expand Up @@ -300,7 +297,7 @@ private static MultiMap getHeaders(BidRequest request) {
public Result<List<BidderBid>> makeBids(BidderCall<Void> httpCall, BidRequest bidRequest) {
final List<NameValuePair> params;
try {
params = URLEncodedUtils.parse(new URI(httpCall.getRequest().getUri()), StandardCharsets.UTF_8);
params = new URIBuilder(httpCall.getRequest().getUri()).getQueryParams();
} catch (URISyntaxException e) {
return Result.withError(BidderError.badInput(e.getMessage()));
}
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/prebid/server/bidder/amx/AmxBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
import com.iab.openrtb.response.SeatBid;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.client.utils.URIBuilder;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.bidder.Bidder;
import org.prebid.server.bidder.amx.model.AmxBidExt;
import org.prebid.server.bidder.model.BidderBid;
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/prebid/server/bidder/appnexus/AppnexusBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
import org.apache.commons.lang3.BooleanUtils;
import org.apache.commons.lang3.ObjectUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.client.utils.URIBuilder;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.auction.model.Endpoint;
import org.prebid.server.bidder.Bidder;
import org.prebid.server.bidder.appnexus.proto.AppnexusBidExt;
Expand Down
15 changes: 10 additions & 5 deletions src/main/java/org/prebid/server/bidder/audiencenetwork/AudienceNetworkBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,23 +53,26 @@ public class AudienceNetworkBidder implements Bidder<BidRequest> {

private static final List<Integer> SUPPORTED_BANNER_HEIGHT = Arrays.asList(250, 50);

private static final String PLATFORM_MACRO = "{{PlatformId}}";
private static final String REQUEST_MACRO = "{{RequestId}}";
private static final String PUBLISHER_MACRO = "{{PublisherId}}";

private final String endpointUrl;
private final String platformId;
private final String appSecret;
private final String timeoutNotificationUrlTemplate;
private final String timeoutNotificationUrl;
private final JacksonMapper mapper;

public AudienceNetworkBidder(String endpointUrl,
String platformId,
String appSecret,
String timeoutNotificationUrlTemplate,
String timeoutNotificationUrl,
JacksonMapper mapper) {

this.endpointUrl = HttpUtil.validateUrl(Objects.requireNonNull(endpointUrl));
this.platformId = checkBlankString(Objects.requireNonNull(platformId), "platform-id");
this.appSecret = checkBlankString(Objects.requireNonNull(appSecret), "app-secret");
this.timeoutNotificationUrlTemplate = HttpUtil.validateUrl(
Objects.requireNonNull(timeoutNotificationUrlTemplate));
this.timeoutNotificationUrl = HttpUtil.validateUrl(Objects.requireNonNull(timeoutNotificationUrl));
this.mapper = Objects.requireNonNull(mapper);
}

Expand Down Expand Up @@ -343,7 +346,9 @@ public HttpRequest<Void> makeTimeoutNotification(HttpRequest<BidRequest> httpReq

return HttpRequest.<Void>builder()
.method(HttpMethod.GET)
.uri(timeoutNotificationUrlTemplate.formatted(platformId, publisherId, requestId))
.uri(timeoutNotificationUrl.replace(PLATFORM_MACRO, HttpUtil.encodeUrl(platformId))
.replace(PUBLISHER_MACRO, HttpUtil.encodeUrl(publisherId))
.replace(REQUEST_MACRO, HttpUtil.encodeUrl(requestId)))
.build();
}
}
2 changes: 1 addition & 1 deletion src/main/java/org/prebid/server/bidder/bluesea/BlueSeaBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import com.iab.openrtb.response.BidResponse;
import com.iab.openrtb.response.SeatBid;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.http.client.utils.URIBuilder;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.bidder.Bidder;
import org.prebid.server.bidder.model.BidderBid;
import org.prebid.server.bidder.model.BidderCall;
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/prebid/server/bidder/connatix/ConnatixBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.lang3.ObjectUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.client.utils.URIBuilder;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.bidder.Bidder;
import org.prebid.server.bidder.model.BidderBid;
import org.prebid.server.bidder.model.BidderCall;
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/prebid/server/bidder/dxkulture/DxKultureBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import com.iab.openrtb.response.SeatBid;
import io.vertx.core.MultiMap;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.http.client.utils.URIBuilder;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.bidder.Bidder;
import org.prebid.server.bidder.model.BidderBid;
import org.prebid.server.bidder.model.BidderCall;
Expand Down
13 changes: 9 additions & 4 deletions src/main/java/org/prebid/server/bidder/emxdigital/EmxDigitalBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
import org.prebid.server.util.HttpUtil;

import java.math.BigDecimal;
import java.time.Instant;
import java.time.Clock;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
Expand All @@ -41,16 +41,20 @@ public class EmxDigitalBidder implements Bidder<BidRequest> {

private static final String USD_CURRENCY = "USD";
private static final Integer PROTOCOL_VAST_40 = 7;
private static final String URL_TIMEOUT_MACRO = "{{URL_TIMEOUT}}";
private static final String TIMESTAMP_MACRO = "{{TIMESTAMP}}";

private static final TypeReference<ExtPrebid<?, ExtImpEmxDigital>> EMXDIGITAL_EXT_TYPE_REFERENCE =
new TypeReference<>() {
};

private final String endpointUrl;
private final Clock clock;
private final JacksonMapper mapper;

public EmxDigitalBidder(String endpointUrl, JacksonMapper mapper) {
public EmxDigitalBidder(String endpointUrl, Clock clock, JacksonMapper mapper) {
this.endpointUrl = HttpUtil.validateUrl(Objects.requireNonNull(endpointUrl));
this.clock = Objects.requireNonNull(clock);
this.mapper = Objects.requireNonNull(mapper);
}

Expand Down Expand Up @@ -231,8 +235,9 @@ private String makeUrl(BidRequest bidRequest) {
final Long tmax = bidRequest.getTmax();
final int urlTimeout = tmax == 0 ? 1000 : tmax.intValue();

return "%s?t=%s&ts=%s&src=pbserver"
.formatted(endpointUrl, urlTimeout, (int) Instant.now().getEpochSecond());
return endpointUrl
.replace(URL_TIMEOUT_MACRO, String.valueOf(urlTimeout))
.replace(TIMESTAMP_MACRO, String.valueOf((int) clock.millis()));
}

@Override
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/org/prebid/server/bidder/eplanning/EplanningBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.lang3.ObjectUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.client.utils.URIBuilder;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.bidder.Bidder;
import org.prebid.server.bidder.eplanning.model.CleanStepName;
import org.prebid.server.bidder.eplanning.model.HbResponse;
Expand Down Expand Up @@ -286,7 +286,7 @@ private String resolveRequestUri(BidRequest request, List<String> requestsString

private static URL parseUrl(String url) {
try {
return new URL(url);
return HttpUtil.parseUrl(url);
} catch (MalformedURLException e) {
throw new PreBidException("Invalid url: " + url, e);
}
Expand Down
27 changes: 18 additions & 9 deletions src/main/java/org/prebid/server/bidder/flipp/FlippBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@
import io.vertx.core.http.HttpMethod;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.NameValuePair;
import org.apache.http.client.utils.URLEncodedUtils;
import org.apache.hc.core5.http.NameValuePair;
import org.apache.hc.core5.net.URIBuilder;
import org.prebid.server.bidder.Bidder;
import org.prebid.server.bidder.flipp.model.request.CampaignRequestBody;
import org.prebid.server.bidder.flipp.model.request.CampaignRequestBodyUser;
Expand Down Expand Up @@ -47,7 +47,7 @@
import org.prebid.server.util.HttpUtil;
import org.prebid.server.util.ObjectUtil;

import java.nio.charset.StandardCharsets;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
Expand Down Expand Up @@ -168,12 +168,21 @@ private static String resolveContentCode(Site site, ExtImpFlipp extImp) {
.map(Site::getPage)
.orElse(null);

return URLEncodedUtils.parse(pageUrl, StandardCharsets.UTF_8)
.stream()
.filter(nameValuePair -> nameValuePair.getName().contains("flipp-content-code"))
.map(NameValuePair::getValue)
.findFirst()
.orElse(null);
if (pageUrl == null) {
return null;
}

try {
return new URIBuilder(pageUrl)
.getQueryParams()
.stream()
.filter(nameValuePair -> nameValuePair.getName().contains("flipp-content-code"))
.map(NameValuePair::getValue)
.findFirst()
.orElse(null);
} catch (URISyntaxException e) {
return null;
}
}

private static List<String> resolveKeywords(BidRequest bidRequest) {
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/prebid/server/bidder/missena/MissenaBidder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ private MultiMap makeHeaders(Device device, Site site) {
if (site != null && StringUtils.isNotBlank(site.getPage())) {
HttpUtil.addHeaderIfValueIsNotEmpty(headers, HttpUtil.REFERER_HEADER, site.getPage());
try {
final URL url = new URL(site.getPage());
final URL url = HttpUtil.parseUrl(site.getPage());
final String origin = url.getProtocol() + "://" + url.getHost();
HttpUtil.addHeaderIfValueIsNotEmpty(headers, HttpUtil.ORIGIN_HEADER, origin);
} catch (MalformedURLException e) {
Expand Down
Loading
Loading