Skip to content

Commit f20d731

Browse files
#1529 added first implementation of extensible artifact search REST API (#1556)
* #1529 added first implementation of extensible artifact search REST API added new getMavenArtifactsByGroupId to MavenUtils which takes a repository type (e.g. maven, nexus, jfrog) and a groupId to search for added new SearchResponse tos to core-api utils added new MavenUtilTest class (tests for proper json string parsing and REST API requests) and resources * #1529 added missing dependencies * #1529 implemented requested changes added new RESTSearchResponseException replaced IOException with RESTSearchResponseException * #1529 fixed jFrog API * #1529 implemented requested changes changed repositoryType from String to MavenSearchRepositoryType enum added new MavenSearchRepositoryType enum * #1529 implemented requested changes replaced fixed repository URLs with constants added new MavenSearchRepositoryConstants * #1529 implemented requested changes converted MavenSearchResponseConstants to uppercase added MAVEN, NEXUS and JFROG TARGET_LINK constants added MAVEN_MAX_RESPONSE_ROWS constant added javadoc to all models (moved main model to top) renamed json response models * #1529 adjusted nexus REST API adjusted nexus REST API to v2.0 adjusted nexus REST API tests and resources to v2.0 refactored createDownloadLink method (moved to AbstractRESTSearchResponse) added new constants for MAVEN_REPOSITORY_LINK, NEXUS_REPOSITORY_URL, NEXUS_REPOSITORY_URL, NEXUS_TARGET_LINK, NEXUS_DC_ID added ec param to MavenSearchResponse * #1529 disabled jfrog REST API test * #1529 added optimizations + authentication added bearer token authentication added bearer token to jfrog renamed nexus to nexus2 repository type added nexus3 repository type added more tests for code coverage * #1529 applied factory pattern added new SearchResponse interface made all SearchResponse types inherit from SearchResponse added new SearchResponseFactory refactored getArtifactDownloadLinks method added new getAvailableSearchInterfaces method (used to register new search interfaces) added getRepositoryType to SearchResponse parent class (returns the type of repository as an enum) * #1529 moved utility methods from factory class to util class added new SearchResponseUtil class * changed javadoc * #1529 implemented requested changes renamed RESTSearchResponse signature msg to message * #1529 implemented requested changes removed limitRows functionality removed NEXUS2_DC_ID from target link adjusted MavenSearchResponseConstants javadoc * #1529 implemented requested changes converted getAvailableSearchInterfaces method to final list of SearchResponses adjusted javadoc * #1529 added missing javadoc * #1529 implemented requested changes added more detailed javadoc * #1529 implemented requested changes converted RESTSearchResponseException name to PascalCase * #1529 implemented requested changes renamed getMavenArtifactsByGroupId to retrieveMavenArtifactsByGroupId * #1529 implemented requested changes renamed getArtifactDownloadLinks to searchArtifactDownloadLinks * #1529 implemented requested changes moved multi search response classes to separate packages * #1529 implemented requested changes improved javadoc readability * #1529 implemented requested changes converted SearchResponse to AbstractSearchResponse refactored getJsonResponse added new retrieveJsonResponseWithAuthentication method to AbstractSearchResponse renamed getJsonResponse to retrieveJsonResponse * #1529 implemented requested changes refactored getDownloadURLs method added new removeDuplicatedDownloadURLs method to AbstractSearchResponse renamed getDownloadURLs to retrieveDownloadURLs * #1529 implemented requested changes refactored SearchResponseUtil methods moved SearchResponseUtil methods to AbstractSearchResponse removed SearchResponseUtil class * #1529 implemented requested changes removed throw CobiGenRuntimeExceptions to ensure that an error with the API won't stop CobiGen execution replaced CobiGenRuntimeException in MavenUtil with error log and a return of null converted testWrongRepositoryTypeThrowsException to testRetrieveMavenArtifactsWithInvalidLinkReturnsNull replaced throw exception with error log message * #1529 implemented requested changes refactored getJsonResponseStringByTargetLink replaced getJsonResponseStringByTargetLink with retrieveJsonResponseWithAuthenticationToken added MavenSearchRepositoryType to retrieveJsonResponseWithAuthenticationToken * #1529 implemented requested changes replaced fixed ignored json properties with ignore unknown param * #1529 added WireMock to tests made ignored tests functional with WireMock added WireMock stubs for each case updated jersey from 3.0.5 to 3.0.7 added wiremock-standalone 2.27.2 (used older version because of conflicts with jackson) * #1529 reduced WireMock logging added logback-test.xml (sets WireMock to WARN log level) * #1529 updated jackson updated jackson-databind from 2.13.2.2 to 2.13.3 * #1529 fixed FileUtils issue replaced readString with readAllBytes * #1529 implemented requested changes renamed baseURL to baseUrl removed LOG concatenation * #1529 implemented requested changes removed ProcessingException from SearchResponseFactory * #1529 implemented requested changes added artifactory path to jfrog target link constant * #1529 removed jersey dependencies replaced jersey with OkHttpClient restricted MavenUtilTests (added check of messages) added jackson-databind to core-api dependencies added okhttp to core-api dependencies * #1529 updated okhttp changed okhttp version from 4.9.1 to 4.10.0 * #1529 added okhttp version to root pom added latest okhttp dependeny to root pom.xml removed fixed okhttp versions from core-api and core-externalprocess-api * #1529 added okio added transitive okio dependency * #1529 implemented requested changes Changed enum values to uppercase * #1529 implemented requested changes moved status into not null if condition Co-authored-by: EduardKrieger <[email protected]>
1 parent 2ee15d3 commit f20d731

27 files changed

+1666
-3
lines changed

cobigen/cobigen-core-api/pom.xml

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
<groupId>com.google.guava</groupId>
2828
<artifactId>guava</artifactId>
2929
</dependency>
30-
30+
3131
<!-- Proper process reader -->
3232
<dependency>
3333
<groupId>org.zeroturnaround</groupId>
@@ -40,11 +40,32 @@
4040
<artifactId>core-test</artifactId>
4141
<scope>test</scope>
4242
</dependency>
43+
4344
<dependency>
4445
<groupId>commons-io</groupId>
4546
<artifactId>commons-io</artifactId>
4647
<scope>test</scope>
4748
</dependency>
49+
50+
<!-- enables JSON creation for REST search response API -->
51+
<dependency>
52+
<groupId>com.fasterxml.jackson.core</groupId>
53+
<artifactId>jackson-databind</artifactId>
54+
</dependency>
55+
56+
<!-- http client -->
57+
<dependency>
58+
<groupId>com.squareup.okhttp3</groupId>
59+
<artifactId>okhttp</artifactId>
60+
</dependency>
61+
62+
<!-- Needed for WireMock test support -->
63+
<dependency>
64+
<groupId>com.github.tomakehurst</groupId>
65+
<artifactId>wiremock-standalone</artifactId>
66+
<version>2.27.2</version>
67+
<scope>test</scope>
68+
</dependency>
4869
</dependencies>
4970

5071
<build>
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
package com.devonfw.cobigen.api.constants;
2+
3+
/**
4+
* Constants needed for handling the maven search REST APIs
5+
*/
6+
public class MavenSearchRepositoryConstants {
7+
8+
/**
9+
* Maven repository URL
10+
*/
11+
public static String MAVEN_REPOSITORY_URL = "https://search.maven.org";
12+
13+
/**
14+
* Maven repository download link
15+
*/
16+
public static String MAVEN_REPOSITORY_DOWNLOAD_LINK = "https://repo1.maven.org/maven2";
17+
18+
/**
19+
* Maven target link
20+
*/
21+
public static String MAVEN_TARGET_LINK = "solrsearch/select";
22+
23+
/**
24+
* Nexus2 repository URL
25+
*/
26+
public static String NEXUS2_REPOSITORY_URL = "https://s01.oss.sonatype.org";
27+
28+
/**
29+
* Nexus2 repository link
30+
*/
31+
public static String NEXUS2_REPOSITORY_LINK = "service/local/repositories/releases/content";
32+
33+
/**
34+
* Nexus2 target link
35+
*/
36+
public static String NEXUS2_TARGET_LINK = "service/local/lucene/search";
37+
38+
/**
39+
* Nexus3 target link
40+
*/
41+
public static String NEXUS3_TARGET_LINK = "service/rest/v1/search";
42+
43+
/**
44+
* Nexus3 repository URL
45+
*/
46+
public static String NEXUS3_REPOSITORY_URL = "";
47+
48+
/**
49+
* Jfrog repository URL
50+
*/
51+
public static String JFROG_REPOSITORY_URL = "http://localhost:8082/artifactory";
52+
53+
/**
54+
* Jfrog target link
55+
*/
56+
public static String JFROG_TARGET_LINK = "artifactory/api/search/gavc";
57+
58+
}
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
package com.devonfw.cobigen.api.constants;
2+
3+
/**
4+
* Maven search repository types used to identify and name the available search REST API types (add new search
5+
* repository types/versions here)
6+
*/
7+
public enum MavenSearchRepositoryType {
8+
9+
/**
10+
* Nexus2 search repository type
11+
*/
12+
NEXUS2,
13+
14+
/**
15+
* Nexus3 search repository type
16+
*/
17+
NEXUS3,
18+
19+
/**
20+
* Maven search repository type
21+
*/
22+
MAVEN,
23+
24+
/**
25+
* Jfrog search repository type
26+
*/
27+
JFROG
28+
}
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
package com.devonfw.cobigen.api.exception;
2+
3+
/** Exception to indicate that the REST search API encountered a problem while accessing the server. */
4+
public class RestSearchResponseException extends CobiGenRuntimeException {
5+
6+
private static final long serialVersionUID = 1L;
7+
8+
/**
9+
* Creates a new {@link RestSearchResponseException} with the given message
10+
*
11+
* @param message error message of the exception
12+
*/
13+
public RestSearchResponseException(String message) {
14+
15+
super(message);
16+
}
17+
18+
/**
19+
* Creates a new {@link RestSearchResponseException} with the specified message and the causing {@link Throwable}
20+
*
21+
* @param message describing the exception
22+
* @param cause the causing Throwable
23+
*/
24+
public RestSearchResponseException(String message, Throwable cause) {
25+
26+
super(message, cause);
27+
}
28+
29+
/**
30+
* Creates a new {@link RestSearchResponseException} with the specified message and the causing {@link Throwable}
31+
*
32+
* @param message describing the exception
33+
* @param statusCode status code causing the {@link RestSearchResponseException} or null if not available
34+
*/
35+
public RestSearchResponseException(String message, String statusCode) {
36+
37+
super(message + statusCode);
38+
}
39+
40+
}

cobigen/cobigen-core-api/src/main/java/com/devonfw/cobigen/api/util/MavenUtil.java

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,9 @@
2727

2828
import com.devonfw.cobigen.api.constants.MavenConstants;
2929
import com.devonfw.cobigen.api.exception.CobiGenRuntimeException;
30+
import com.devonfw.cobigen.api.exception.RestSearchResponseException;
31+
import com.devonfw.cobigen.api.util.to.SearchResponseFactory;
32+
import com.fasterxml.jackson.core.JsonProcessingException;
3033
import com.google.common.collect.Lists;
3134
import com.google.common.hash.Hashing;
3235
import com.google.common.io.ByteSource;
@@ -340,4 +343,27 @@ public static Path getProjectRoot(Path inputFile, boolean topLevel) {
340343
LOG.debug("Project root could not be found.");
341344
return null;
342345
}
346+
347+
/**
348+
* Retrieves a list of download URLs by groupId from the specified repository search REST API using authentication
349+
* with bearer token
350+
*
351+
* @param baseUrl String of the repository server URL
352+
* @param groupId the groupId to search for
353+
* @param authToken bearer token to use for authentication
354+
* @return List of artifact download URLS or null if an error occurred
355+
*/
356+
public static List<URL> retrieveMavenArtifactsByGroupId(String baseUrl, String groupId, String authToken) {
357+
358+
try {
359+
360+
return SearchResponseFactory.searchArtifactDownloadLinks(baseUrl, groupId, authToken);
361+
} catch (RestSearchResponseException | JsonProcessingException | MalformedURLException e) {
362+
LOG.error("Unable to get artifacts from {} by groupId {}", baseUrl, groupId, e);
363+
// TODO: Handle Eclipse, CLI and MavenPlugin here (f.e. with a new Exception)
364+
return null;
365+
}
366+
367+
}
368+
343369
}
Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
package com.devonfw.cobigen.api.util.to;
2+
3+
import java.io.IOException;
4+
import java.net.MalformedURLException;
5+
import java.net.URL;
6+
import java.util.List;
7+
import java.util.concurrent.TimeUnit;
8+
import java.util.stream.Collectors;
9+
10+
import org.slf4j.Logger;
11+
import org.slf4j.LoggerFactory;
12+
13+
import com.devonfw.cobigen.api.constants.MavenSearchRepositoryType;
14+
import com.devonfw.cobigen.api.exception.RestSearchResponseException;
15+
import com.fasterxml.jackson.annotation.JsonIgnore;
16+
import com.fasterxml.jackson.annotation.JsonProperty;
17+
18+
import okhttp3.OkHttpClient;
19+
import okhttp3.Request;
20+
import okhttp3.Response;
21+
22+
/**
23+
* This interface should be inherited for all maven REST search API responses to properly convert {@link JsonProperty}
24+
* from responses to valid download URLs
25+
*/
26+
public abstract class AbstractSearchResponse {
27+
28+
/** Logger instance. */
29+
@JsonIgnore
30+
private static final Logger LOG = LoggerFactory.getLogger(AbstractSearchResponse.class);
31+
32+
/**
33+
* @return the {@link MavenSearchRepositoryType} type
34+
*/
35+
public abstract MavenSearchRepositoryType getRepositoryType();
36+
37+
/**
38+
* Creates a list of download URLs
39+
*
40+
* @return List of download links
41+
* @throws MalformedURLException if an URL was not valid
42+
*/
43+
public abstract List<URL> retrieveDownloadURLs() throws MalformedURLException;
44+
45+
/**
46+
* Removes duplicates from list of download URLs
47+
*
48+
* @param downloadUrls list of download URLs
49+
* @return List of download links
50+
* @throws MalformedURLException if an URL was not valid
51+
*/
52+
public List<URL> removeDuplicatedDownloadURLs(List<URL> downloadUrls) throws MalformedURLException {
53+
54+
return downloadUrls.stream().distinct().collect(Collectors.toList());
55+
}
56+
57+
/**
58+
* Retrieves the json response from a repository URL and a group ID
59+
*
60+
* @param repositoryUrl URL of the repository
61+
* @param groupId to search for
62+
* @return String of json response
63+
* @throws RestSearchResponseException if the request did not return status 200
64+
*/
65+
public String retrieveJsonResponse(String repositoryUrl, String groupId) throws RestSearchResponseException {
66+
67+
return retrieveJsonResponse(repositoryUrl, groupId, null);
68+
}
69+
70+
/**
71+
* Retrieves the json response from a repository URL, a group ID and a bearer authentication token
72+
*
73+
* @param repositoryUrl URL of the repository
74+
* @param groupId to search for
75+
* @param authToken bearer token to use for authentication
76+
* @return String of json response
77+
* @throws RestSearchResponseException if the request did not return status 200
78+
*/
79+
public abstract String retrieveJsonResponse(String repositoryUrl, String groupId, String authToken)
80+
throws RestSearchResponseException;
81+
82+
/**
83+
* Creates a @WebTarget with provided authentication token
84+
*
85+
* @param targetLink link to get response from
86+
* @param token bearer token to use for authentication
87+
* @return Request to use as resource
88+
*/
89+
private static Request bearerAuthenticationWithOAuth2AtClientLevel(String targetLink, String token) {
90+
91+
return new Request.Builder().url(targetLink).addHeader("Authorization", "Bearer " + token).build();
92+
93+
}
94+
95+
/**
96+
* Retrieves a json response by given REST API target link using bearer authentication token
97+
*
98+
* @param targetLink link to get response from
99+
* @param authToken bearer token to use for authentication
100+
* @param searchRepositoryType the type of the search repository
101+
* @return String of json response
102+
* @throws RestSearchResponseException if the returned status code was not 200 OK
103+
*/
104+
public static String retrieveJsonResponseWithAuthenticationToken(String targetLink, String authToken,
105+
MavenSearchRepositoryType searchRepositoryType) throws RestSearchResponseException {
106+
107+
LOG.info("Starting {} search REST API request with URL: {}.", searchRepositoryType, targetLink);
108+
109+
OkHttpClient httpClient = new OkHttpClient().newBuilder().connectTimeout(10, TimeUnit.SECONDS)
110+
.readTimeout(30, TimeUnit.SECONDS).callTimeout(30, TimeUnit.SECONDS).writeTimeout(30, TimeUnit.SECONDS)
111+
.retryOnConnectionFailure(true).build();
112+
String jsonResponse = "";
113+
114+
try {
115+
Response response = null;
116+
117+
if (authToken != null) {
118+
response = httpClient.newCall(bearerAuthenticationWithOAuth2AtClientLevel(targetLink, authToken)).execute();
119+
} else {
120+
response = httpClient.newCall(new Request.Builder().url(targetLink).get().build()).execute();
121+
}
122+
123+
if (response != null) {
124+
int status = response.code();
125+
if (status == 200 || status == 201 || status == 204) {
126+
jsonResponse = response.body().string();
127+
} else {
128+
throw new RestSearchResponseException("The search REST API returned the unexpected status code: ",
129+
String.valueOf(response.code()));
130+
}
131+
}
132+
133+
} catch (IOException e) {
134+
throw new RestSearchResponseException("Unable to send or receive the message from the service", e);
135+
} catch (IllegalArgumentException e) {
136+
throw new RestSearchResponseException("The target URL was faulty.", e);
137+
}
138+
139+
return jsonResponse;
140+
141+
}
142+
143+
/**
144+
* Creates a download link (concatenates maven repository link with groupId, artifact and version)
145+
*
146+
* @param mavenRepo link to the maven repository to use
147+
* @param groupId for the download link
148+
* @param artifactId for the download link
149+
* @param version for the download link
150+
* @param fileEnding file ending for the download link
151+
* @return concatenated download link
152+
* @throws MalformedURLException if the URL was not valid
153+
*/
154+
protected static URL createDownloadLink(String mavenRepo, String groupId, String artifactId, String version,
155+
String fileEnding) throws MalformedURLException {
156+
157+
String parsedGroupId = groupId.replace(".", "/");
158+
String downloadFile = artifactId + "-" + version + fileEnding;
159+
String downloadLink = mavenRepo + "/" + parsedGroupId + "/" + artifactId + "/" + version + "/" + downloadFile;
160+
URL url = new URL(downloadLink);
161+
return url;
162+
}
163+
164+
}

0 commit comments

Comments
 (0)