Hot questions for Using Azure in http

Question:

I deployed spring boot Rest application in azure tomcat app service. Here I copied application from my local tomcat webapps folder to azure tomcat webapps fodler using FTP( copied app folder structure, not war file).

Its working fine.. but after some time server is not responding and returning HTTP Error 502.3 - Bad Gateway.

What could be the reason for this error. Is there any issue with my spring configuration ? appreciate any help.

we have two kind of URI's, one health check URI and another one is to fetch the data from database.

When app service is down - only myapp/healthcheck is returning status code 200.

And remaining all requests which having uri - myapp/ssp/*** returning status code 502.3

When I restart the app service, every thing working, but after some time issue reoccurring.

I am suspecting problem in my AuthenticationFilter.java & RequestValidator. java & ControllerAdvice files

Here is spring boot configuration

here is my configuration. `@SpringBootApplication( exclude = HibernateJpaAutoConfiguration.class ) @ComponentScan( { "com.xx.ssp" , "com.microsoft.applicationinsights" } ) public class SSPApplication{

/**
 * @param args
 */
public static void main( String[ ] args ) {

    SpringApplication sspSpringBootApplciation = new SpringApplication( SSPApplication.class );
    sspSpringBootApplciation.addListeners( new ApplicationPidFileWriter( ) );
    sspSpringBootApplciation.run( args );
}

/**
 * @return
 */
@Bean
public WebSecurityConfigurerAdapter webSecurityConfigurerAdapter( ) {

    return new ApplicationSecurity( );
}

@Bean
public String telemetryConfig( ) {

    String telemetryKey = "zzzz";
    if( telemetryKey != null ){
        TelemetryConfiguration.getActive( ).setInstrumentationKey( telemetryKey );
    }
    return telemetryKey;
}

@Bean
public FilterRegistrationBean aiFilterRegistration( ) {

    FilterRegistrationBean registration = new FilterRegistrationBean( );
    registration.setFilter( webRequestTrackingFilter( ) );
    registration.addUrlPatterns( "/*" );
    registration.setOrder( 1 );
    return registration;
}

@Bean( name = "WebRequestTrackingFilter" )
public Filter webRequestTrackingFilter( ) {

    return new WebRequestTrackingFilter( );
}

@Bean
public ErrorPageFilter errorPageFilter( ) {

    return new ErrorPageFilter( );
}

@Bean
public FilterRegistrationBean disableSpringBootErrorFilter( ErrorPageFilter filter ) {

    FilterRegistrationBean filterRegistrationBean = new FilterRegistrationBean( );
    filterRegistrationBean.setFilter( filter );
    filterRegistrationBean.setEnabled( false );
    return filterRegistrationBean;
}

} `

`@ControllerAdvice public class RestEntityExceptionHandler extends ResponseEntityExceptionHandler{

@Autowired
private Environment environment;

private SSPLogger logger = new SSPLogger( getClass( ) );

/**
 * @return
 */
private String getEnvironment( ) {

    String[ ] activeProfiles = environment.getActiveProfiles( );
    if( activeProfiles != null && activeProfiles.length > 1 ){
        return SSPEnvironment.getEnvironment( activeProfiles[ 0 ] ).name( );
    }
    return SSPEnvironment.UNKNOWN.name( );

}

/**
 * @param errorCode
 * @return
 */
private HttpStatus getHttpstatusCode( String errorCode ) {

    try{
        return HttpStatus.valueOf( Integer.parseInt( errorCode ) );
    } catch ( Exception e ){

        return HttpStatus.INTERNAL_SERVER_ERROR;
    }
}

/**
 * @param ex
 * @return
 */
@ExceptionHandler( {    .class , IllegalStateException.class , SSPRestException.class ,
        Exception.class } )
@ResponseBody
ResponseEntity< Object > handleControllerException( Exception ex ) {

    HttpHeaders headers = new HttpHeaders( );
    headers.setContentType( MediaType.APPLICATION_JSON );
    ex.printStackTrace( );
    if( ex instanceof SSPRestException ){
        SSPRestException exp = ( SSPRestException ) ex;
        CustomErrorResponse customErrorResponse = new CustomErrorResponse( );
        customErrorResponse.setStatusCode( exp.getErrorCode( ) );
        customErrorResponse.setErrorMessage( exp.getErrorMessage( ) );
        customErrorResponse.setErrorDescription( exp.getErrorDescription( ) );
        customErrorResponse.setErrorKey( exp.getErrorKey( ) );
        customErrorResponse.setEnvironment( getEnvironment( ) );
        logger.logException( LogLevel.ERROR , customErrorResponse );

        HttpStatus httpstatusCode = getHttpstatusCode( exp.getErrorCode( ) );

        return new ResponseEntity<>( customErrorResponse , headers , httpstatusCode );
    }
    if( ex instanceof HttpRequestMethodNotSupportedException ){
        String statusCode = "405";
        headers.setContentType( MediaType.APPLICATION_JSON );
        CustomErrorResponse customErrorResponse = new CustomErrorResponse( );
        customErrorResponse.setStatusCode( statusCode );
        customErrorResponse.setErrorMessage( ex.getMessage( ) );
        customErrorResponse.setErrorKey( "" );
        customErrorResponse.setEnvironment( getEnvironment( ) );
        HttpStatus httpstatusCode = getHttpstatusCode( statusCode );
        return new ResponseEntity<>( customErrorResponse , headers , httpstatusCode );
    } else{
        String statusCode = "UNKNOWN";
        CustomErrorResponse customErrorResponse = new CustomErrorResponse( );
        customErrorResponse.setStatusCode( statusCode );
        customErrorResponse.setErrorMessage( ex.getMessage( ) );
        if( ex.getCause( ) != null ){
            customErrorResponse.setErrorDescription( ex.getCause( ).getMessage( ) );
        }
        customErrorResponse.setErrorKey( "UNAHNDLED_EXCEPTION" );
        customErrorResponse.setEnvironment( getEnvironment( ) );
        HttpStatus httpstatusCode = getHttpstatusCode( statusCode );
        logger.logException( LogLevel.ERROR , customErrorResponse );
        return new ResponseEntity<>( customErrorResponse , headers , httpstatusCode );
    }
}

}` ``` @Order( SecurityProperties.ACCESS_OVERRIDE_ORDER ) public class ApplicationSecurity extends WebSecurityConfigurerAdapter{

    @Override
    protected void configure( HttpSecurity http ) throws Exception {

    http.csrf( ).disable( ).sessionManagement( ).sessionCreationPolicy( SessionCreationPolicy.STATELESS );
    http.authorizeRequests( ).antMatchers( "/spp/**" ).addFilterBefore( new AuthenticationFilter( ) , SecurityContextPersistenceFilter.class );
http.authorizeRequests( ).antMatchers( "/healthcheck/**" , "/unoauth/**" ).permitAll( ).antMatchers( HttpMethod.OPTIONS , "/**" ).permitAll( );

```

`@Component public class RequestValidator extends HandlerInterceptorAdapter{

@Autowired
private AppConstants appConstants;

@Autowired
private CacheTemplate cacheTemplate;

@Autowired
private RestClient restClient;

/*
 * (non-Javadoc)
 * @see org.springframework.web.servlet.handler.HandlerInterceptorAdapter#
 * afterCompletion(javax.servlet.http.HttpServletRequest,
 * javax.servlet.http.HttpServletResponse, java.lang.Object,
 * java.lang.Exception)
 */
@Override
public void afterCompletion( HttpServletRequest request , HttpServletResponse response , Object object ,
                Exception arg3 ) throws Exception {

}

/**
 * @param cookies
 * @return
 */
private String getGsidFromRequest( Cookie[ ] cookies ) {

    String gsid = null;
    if( cookies != null ){
        Cookie gsidCookie = Arrays.stream( cookies ).filter( x-> x.getName( ).equals( "gsid" ) ).findFirst( )
                        .orElse( null );
        if( gsidCookie != null ){
            gsid = gsidCookie.getValue( );
        }
        if( Utils.isEmpty( gsid ) ){
            throw new SSPRestException( ErrorCode.InvalidRequest , "gsid is missing" , getClass( ) );
        }
    }
    return gsid;
}

/**
 * @return
 */
private UserCashedInfo getTempUserInfo( ) {

    UserCashedInfo cashedInfo = new UserCashedInfo( );
    cashedInfo.setEmailId( "sssssss" );
    cashedInfo.setUserId( 1 );
    cashedInfo.setUserRole( 1 );
    cashedInfo.setRefreshToken( "" );
    cashedInfo.setCaseInstanceId( 1 );
    cashedInfo.setUserName( "Test User" );
    return cashedInfo;
}

/**
 * @param accessToken
 * @return
 */
private boolean isRequestHadAccessToken( String accessToken ) {

    if( Utils.isEmpty( accessToken ) ){
        throw new SSPRestException( ErrorCode.InvalidRequest , "Access Token is Mandatory" , getClass( ) );
    }
    return true;
}

/*
 * (non-Javadoc)
 * @see org.springframework.web.servlet.handler.HandlerInterceptorAdapter#
 * postHandle(javax.servlet.http.HttpServletRequest,
 * javax.servlet.http.HttpServletResponse, java.lang.Object,
 * org.springframework.web.servlet.ModelAndView)
 */
@Override
public void postHandle( HttpServletRequest request , HttpServletResponse response , Object object ,
                ModelAndView model ) throws Exception {

}

/*
 * (non-Javadoc)
 * @see org.springframework.web.servlet.handler.HandlerInterceptorAdapter#
 * preHandle(javax.servlet.http.HttpServletRequest,
 * javax.servlet.http.HttpServletResponse, java.lang.Object)
 */
@Override
public boolean preHandle( HttpServletRequest request , HttpServletResponse response , Object object )
                throws Exception {

    boolean isValidRequest = false;

    String isOauthRequiredParam = request.getParameter( "isOauth" );

    String methodName = request.getMethod( );
    if( methodName != null && methodName.equalsIgnoreCase( "OPTIONS" ) ){
        isValidRequest = true;
    }

    if( !isValidRequest ){
        if( !Utils.isEmpty( isOauthRequiredParam ) && isOauthRequiredParam.equals( "false" ) ){
            isValidRequest = true;
            UserCashedInfo cashedInfo = getTempUserInfo( );
            Utils.updateUserLocalThread( cashedInfo );
        } else{
            String accessToken = request.getHeader( "oauth" );
            boolean isRequestHadAccessToken = isRequestHadAccessToken( accessToken );
            String gsid = getGsidFromRequest( request.getCookies( ) );
            if( isRequestHadAccessToken && !Utils.isEmpty( gsid ) ){
                UserCashedInfo cashedInfo = cacheTemplate.getItem( gsid );
                if( cashedInfo != null ){
                    String userEmail = validateAccessToken( accessToken );
                    if( userEmail != null && userEmail.equals( cashedInfo.getEmailId( ) ) ){
                        Utils.updateUserLocalThread( cashedInfo );
                        isValidRequest = true;
                    } else{
                        throw new SSPRestException( ErrorCode.UnAuthenticateUser , "" , getClass( ) );
                    }

                } else{
                    throw new SSPRestException( ErrorCode.UnAuthenticateUser , "" , getClass( ) );
                }
            } else{
                throw new SSPRestException( ErrorCode.GsidMissing , "" , getClass( ) );
            }
        }
    }
    return isValidRequest;
}

/**
 * @param accessToken
 * @return
 */
private String validateAccessToken( String accessToken ) {

    CustomeJsonResponse validateResponse = restClient.ValidateAcessToken( accessToken );
    if( validateResponse.getStatusCode( ) == 200 ){
        return ( ( ValidateAcessTokenResponse ) validateResponse.getData( ) ).getAccess_token( )
                        .get( appConstants.getUserEmail( ) );
    } else{
        throw new SSPRestException( ErrorCode.ApValidateAccessTokenError , "" , getClass( ) );
    }
}

} `

` public class AuthenticationFilter extends GenericFilter{

private SSPLogger logger = new SSPLogger( getClass( ) );

@Override
public void destroy( ) {

}

/*
 * (non-Javadoc)
 * @see com.xxx.ssp.security.GenericFilter#doFilter(javax.servlet.
 * ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain)
 */
@Override
public void doFilter( ServletRequest req , ServletResponse res , FilterChain chain )
                throws IOException , ServletException {

    HttpServletRequest request = ( HttpServletRequest ) req;

    String accessToken = request.getHeader( "oauth" );
    boolean isValidRequest = false;
    String isOauthRequiredParam = request.getParameter( "isOauth" );
    String methodName = request.getMethod( );
    if( methodName != null && methodName.equalsIgnoreCase( "OPTIONS" ) ){
        isValidRequest = true;
    } else{
        if( !Utils.isEmpty( isOauthRequiredParam ) && isOauthRequiredParam.equals( "false" ) ){
            isValidRequest = true;
        } else{
            logger.debug( "Requeste validation required : true" );

            boolean isRequestHadAccessToken = isRequestHadAccessToken( accessToken );
            String gsid = getGsidFromRequest( request.getCookies( ) );

            if( isRequestHadAccessToken && !Utils.isEmpty( gsid ) ){
                isValidRequest = true;
            } else{
                throw new SSPRestException( ErrorCode.InternalServerError ,
                                " Access Token or GSID is mssing in the request" , getClass( ) );
            }
        }
    }
    if( isValidRequest ){
        chain.doFilter( req , res );
    } else{
        throw new SSPRestException( ErrorCode.UnauthorisedRequest , "Invalid User" , getClass( ) );
    }
}

/**
 * @param cookies
 * @return
 */
private String getGsidFromRequest( Cookie[ ] cookies ) {

    String gsid = null;
    if( cookies != null ){
        Cookie gsidCookie = Arrays.stream( cookies ).filter( x-> x.getName( ).equals( "gsid" ) ).findFirst( )
                        .orElse( null );
        if( gsidCookie != null ){
            gsid = gsidCookie.getValue( );
        }
        if( Utils.isEmpty( gsid ) ){
            throw new SSPRestException( ErrorCode.UnAuthenticateUser , "gsid is missing" , getClass( ) );
        }
    }
    return gsid;
}

/*
 * (non-Javadoc)
 * @see
 * com.xxx.ssp.security.GenericFilter#init(javax.servlet.FilterConfig)
 */
@Override
public void init( FilterConfig arg0 ) throws ServletException {

}

/**
 * @param accessToken
 * @return
 */
private boolean isRequestHadAccessToken( String accessToken ) {

    if( Utils.isEmpty( accessToken ) ){
        throw new SSPRestException( ErrorCode.UnAuthenticateUser , "Access Token is Mandatory" , getClass( ) );
    }
    return true;
}

}`

`@Component public class SSPCORSFilter implements Filter{

private SSPLogger sSPLogger = null;

public SSPCORSFilter( ){

    sSPLogger = new SSPLogger( getClass( ) );
    sSPLogger.info( "CORSFilter initialized" );
}

@Override
public void destroy( ) {

}

@Override
public void doFilter( ServletRequest req , ServletResponse res , FilterChain chain )
                throws IOException , ServletException {

    String origin = ( ( HttpServletRequest ) req ).getHeader( "Origin" );
    sSPLogger.info( "CORSFilter Request origin : " + origin );
    HttpServletResponse response = ( HttpServletResponse ) res;
    response.setHeader( "Access-Control-Allow-Origin" , origin );
    response.setHeader( "Access-Control-Allow-Credentials" , "true" );
    response.setHeader( "Access-Control-Allow-Methods" , "GET, POST, PUT, DELETE, OPTIONS" );
    // response.setHeader( "Access-Control-Allow-Headers" , "Content-Type,
    // Accept, X-Requested-With, oauth" );
    response.setHeader( "Access-Control-Request-Headers" ,
                    "Access-Control-Allow-Headers, Origin,Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers, oauth, access-control-allow-credentials" );
    response.setHeader( "Access-Control-Allow-Headers" ,
                    "Access-Control-Allow-Headers, Origin,Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers, oauth, access-control-allow-credentials" );

    chain.doFilter( req , res );
}

@Override
public void init( FilterConfig filterConfig ) {

}

}`


Answer:

I fixed this issue after looking at thread dump. The root cause is dead lock occurred at db connection pool. So my request not getting DB connection.. after time out IIS server returning my request with status as "http-error-502-3-bad-gateway"

Question:

I was looking for some info on how to deploy Akka Http app to Azure. Azure supports Java Web Apps, but I assume it does only for apps that run in TomCat or alike. But Akka Http is a standalone application - is there any support for that and in any case, how should I proceed with it?

I am aware of sbt-native-packager that can produce various installable formats, but seeing how easy it is to deploy .NET app from Visual Studio, I would love to see a simpler way for Java as well.


Answer:

Based on my understanding, your app using Akka Http is a standalone jar file which need to be deployed on Azure WebApps. Please refer to the section Application configuration Examples of the offical tutorial Upload a custom Java web app to Azure to create a web.xml file in the path wwwroot to deploy your app like these samples Jetty or Springboot.

Question:

I don't know why, but the post that I do with Vertx is simply not working. All the time the error is 404. The same link and body I used with pure Java, and I got the response from the server. What do I do wrong?

HttpClient client = vertx.createHttpClient();

HttpClientRequest request = 
client.post("https://login.windows.net/common/oauth2/token").handler(res->{
                System.out.println(res.statusCode());
            }).putHeader(HttpHeaders.CONTENT_LENGTH,String.valueOf(buffer.length()))
.putHeader(HttpHeaders.CONTENT_TYPE,"application/x-www-form-urlencoded").write(buffer);
            request.end(); 

I am basically doing authentication with Azure, and for the response I should get a JSON with token and other info. With pure Java, works, but we need to make it work with Vertx.

EDIT - this code works - I get JSON back, but is not vertx

String url = "https://login.microsoftonline.com/common/oauth2/token";
URL obj = null;
obj = new URL(url);
HttpsURLConnection con = null;
con = (HttpsURLConnection) obj.openConnection();

//add reuqest header
con.setRequestMethod("POST");
con.setRequestProperty("User-Agent", USER_AGENT);
con.setRequestProperty("Host", "login.microsoftonline.com");
con.setRequestProperty("Cache-Control", "no-cache");
con.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");

// Send post request
con.setDoOutput(true);
DataOutputStream wr = null;

wr = new DataOutputStream(con.getOutputStream());
wr.writeBytes(urlParameters);
wr.flush();
wr.close();

int responseCode = 0;
responseCode = con.getResponseCode();

System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + urlParameters);
System.out.println("Response Code : " + responseCode);

BufferedReader in = null;
in = new BufferedReader(
                    new InputStreamReader(con.getInputStream()));

String inputLine;
StringBuffer response = new StringBuffer();

while ((inputLine = in.readLine()) != null) {
    response.append(inputLine);
}

in.close();

//print result
System.out.println(response.toString());

Answer:

It seems that the issue was caused by requesting a HTTPS url without enable SSL and specify the 443 port. Vert.x httpclient default support HTTP request to access the port 80 of a web host. You need to enable SSL support for httpclient via HttpClientOptions.

Please try to use the code below instead of yours.

HttpClient client = vertx.createHttpClient(new HttpClientOptions().setSsl(true).setTrustAll(true));
HttpClientRequest request = client.post(443, "login.windows.net", "/common/oauth2/token").handler(res->{
                System.out.println(res.statusCode());
     }).putHeader(HttpHeaders.CONTENT_LENGTH,String.valueOf(buffer.length()))
.putHeader(HttpHeaders.CONTENT_TYPE,"application/x-www-form-urlencoded").write(buffer);
request.end(); 

As references, please view the offical doc http://vertx.io/docs/vertx-core/java/#_using_https_with_vert_x and the code sample in GitHub https://github.com/vert-x3/vertx-examples/blob/master/core-examples/src/main/java/io/vertx/example/core/http/https/Client.java.

Question:

   public HttpResponseMessage run(
            @HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.FUNCTION) HttpRequestMessage<Optional<String>> request,
            final ExecutionContext context) {

There's only HttpRequestMessage parameter on the run() method in the Azure Functions(Java) spec. I need to declare and use MultipartHttpServletRequest to fetch a file from the multipart/data request. I'm trying but cannot see any way to cast HttpRequestMessag to MultipartHttpServletRequest.

Please give me some advice.

The HttpTrigger spec is : https://docs.microsoft.com/en-us/java/api/com.microsoft.azure.functions.annotation.httptrigger?view=azure-java-stable

----------------------- update -------------------------

The uploaded image is still corrupted. The size is exaclty same as the original one, but it seems like this :

I will paste the entire code. Please review it.

Function Class source :

public class HttpTriggerJava {
    private static final String storageConnectionString =
            "DefaultEndpointsProtocol=http;" +
                    "AccountName=00000;" +
                    "AccountKey=00000";

    @FunctionName("HttpTriggerJava")
    public HttpResponseMessage run(
            @HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.FUNCTION) HttpRequestMessage<Optional<String>> request,
            final ExecutionContext context) throws Exception{

        context.getLogger().info("Java HTTP trigger processed a request.");

        CloudStorageAccount storageAccount = CloudStorageAccount.parse(storageConnectionString);
        CloudBlobClient blobClient = storageAccount.createCloudBlobClient();
        CloudBlobContainer container = blobClient.getContainerReference("contents");

        // here the "content-type" must be lower-case
        String contentType = request.getHeaders().get("content-type"); // Get content-type header

        String body = request.getBody().get(); // Get request body
        String boundary = contentType.split(";")[1].split("=")[1]; // Get boundary from content-type header
        int bufSize = 1024;
        InputStream in = new ByteArrayInputStream(body.getBytes()); // Convert body to an input stream
        MultipartStream multipartStream  = new MultipartStream(in, boundary.getBytes(), bufSize, null); // Using MultipartStream to parse body input stream
        boolean nextPart = multipartStream.skipPreamble();
        while(nextPart) {
            String header = multipartStream.readHeaders();
            System.out.println("");
            System.out.println("Headers:");
            System.out.println(header);
            System.out.println("Body:");
            if (header.contains("Content-Type: image/")) {
                int start = header.indexOf("filename=")+"filename=".length()+1;
                int end = header.indexOf("\r\n")-1;
                String filename = header.substring(start, end);
                System.out.println(filename);
                FileOutputStream fos = new FileOutputStream(filename);
                multipartStream.readBodyData(fos);

                File sourceFile = new File(filename);
                CloudBlockBlob blob = container.getBlockBlobReference(filename);
                blob.uploadFromFile(sourceFile.getAbsolutePath());

            } else {
                multipartStream.readBodyData(System.out);
            }
            System.out.println("");
            nextPart = multipartStream.readBoundary();
        }

        return request.createResponseBuilder(HttpStatus.OK).body("Success").build();
    }
}

And the HTML is :

<head>
<script src="https://code.jquery.com/jquery-3.3.1.min.js"></script>
<script>
$(document).ready(function () {
    $("#myFile").change(function() {
      readURL(this);
    });

    $("#submit").click(function (event) {
        event.preventDefault();

        var form = $('#form')[0];
        var data = new FormData(form);

        $("#submit").prop("disabled", true);

        $.ajax({
            type: "POST",
            enctype: 'multipart/form-data',
            url: $(form).attr('action'),
            data: data,
            processData: false,
            contentType: false,
            cache: false,
            timeout: 600000,
            success: function (data) {
                $("#result").text(data);
                console.log("SUCCESS : ", data);
                $("#submit").prop("disabled", false);
            },
            error: function (e) {
                $("#result").text(e.responseText);
                console.log("ERROR : ", e);
                $("#submit").prop("disabled", false);
            }
        });
    });
});
function readURL(input) {
  if (input.files && input.files[0]) {
    var reader = new FileReader();

    reader.onload = function(e) {
      $('#blah').attr('src', e.target.result).show();
    }
    reader.readAsDataURL(input.files[0]);
  }
}
</script>
</head>

<body>
    <form id=form
        action="http://doopediafunctiontest.azurewebsites.net/api/HttpTriggerJava?code=00000"
        method="post" enctype="multipart/form-data">
        <p>
            <br /> <br /> <strong>My file:</strong><br /> <input type="file" id="myFile" name="myFile">
            <br /><img id="blah" src="#" alt="your image" style="display:none" />
        </p>
        <input id=submit type="submit" value="upload to Blob Storage">
    </form>

    <div id=result></div>
</body>

I compare the original image and the corrupted image by a hex editor. And I found some random hexes changed to 3f, it should be the reason. Maybe there's some encoding problem. But how can I fix this?

(Please click to enlarge)


Answer:

It sounds like you want to upload a file to your Azure Function with Http Trigger in Java via a HTML form with multipart/form-data like below.

<form method="POST" enctype="multipart/form-data" action="https://<your function app>/api/HttpTrigger-Java">
  File to upload: <input type="file" name="upfile"><br/>
  Notes about the file: <input type="text" name="note"><br/>
  <br/>
  <input type="submit" value="Press"> to upload the file!
</form>

However, there is not any class implements the interface HttpRequestMessage<T> and seems to not cast HttpRequestMessage to HttpServletRequest after I researched the source code of GitHub Repo Azure/azure-functions-java-library.

Per my experience, the only way is to parse the header and body of a multipart/form-data request to get the file. There is an answer of the similar SO thread Library and examples of parsing multipart/form-data from inputstream posted by the question owner, which includes the code using MultipartStream class of Apache Commons FileUpload that works after I test it.

Here is the Content-Type header and body of a multipart/form-data request received from Azure Function for Java.

Header Content-Type

content-type: multipart/form-data; boundary=----WebKitFormBoundaryT2TWuevX3RIYWRQF

multipart/form-data request body

------WebKitFormBoundaryT2TWuevX3RIYWRQF
Content-Disposition: form-data; name="upfile"; filename="z.txt"
Content-Type: text/plain
1234
ABCD
------WebKitFormBoundaryT2TWuevX3RIYWRQF
Content-Disposition: form-data; name="note"
test.txt
------WebKitFormBoundaryT2TWuevX3RIYWRQF--

Here is my sample code to fetch the file.

@FunctionName("HttpTrigger-Java")
public HttpResponseMessage run(
        @HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<String>> request,
        final ExecutionContext context) {
    String contentType = request.getHeaders().get("content-type"); // Get content-type header
    // here the "content-type" must be lower-case
    String body = request.getBody().get(); // Get request body
    InputStream in = new ByteArrayInputStream(body.getBytes()); // Convert body to an input stream
    String boundary = contentType.split(";")[1].split("=")[1]; // Get boundary from content-type header
    int bufSize = 1024;
    MultipartStream multipartStream  = new MultipartStream(in, boundary.getBytes(), bufSize, null); // Using MultipartStream to parse body input stream
    // the code below comes from the SO thread above
    // you can fetch a file content from readBodyData 
    // after the headers Content-Disposition: form-data; name="upfile"; filename="test.txt" \n Content-Type: text/plain
    boolean nextPart = multipartStream.skipPreamble();
    while (nextPart) {
        String header = multipartStream.readHeaders();
        System.out.println("");
        System.out.println("Headers:");
        System.out.println(header);
        System.out.println("Body:");
        multipartStream.readBodyData(System.out);
        System.out.println("");
        nextPart = multipartStream.readBoundary();
    }
    return request.createResponseBuilder(HttpStatus.OK).body("Success").build();
}

The output of code above in terminal :

Headers:
Content-Disposition: form-data; name="upfile"; filename="test.txt"
Content-Type: text/plain


Body:
1234
ABCD


Headers:
Content-Disposition: form-data; name="note"


Body:
test.txt

Update: If upload an image, the output of the code above is like below.

Headers:
Content-Disposition: form-data; name="upfile"; filename="test.jpg"
Content-Type: image/png


Body:
<the binary content of an image>

So you can parse the header to get the filename value to use FileOutputStream to store it, as the code below.

while(nextPart) {
    String header = multipartStream.readHeaders();
    System.out.println("");
    System.out.println("Headers:");
    System.out.println(header);
    System.out.println("Body:");
    if (header.contains("Content-Type: image/")) {
        int start = header.indexOf("filename=")+"filename=".length()+1;
        int end = header.indexOf("\r\n")-1;
        String filename = header.substring(start, end);
        System.out.println(filename);
        FileOutputStream fos = new FileOutputStream(filename);
        multipartStream.readBodyData(fos);
    } else {
        multipartStream.readBodyData(System.out);
    }
    System.out.println("");
    nextPart = multipartStream.readBoundary();
}

Update 2:

I discovered there seems to be an issue of Azure Function for Java which may be a bug that will lose some bytes when uploading binary file, but it will not happend for uploading text file. So a workaround solution is to convert upload file to base64 string in browser to post to Azure Function and convert base64 content uploaded to the origin binary file in Azure Function.

Here is my testing HTML code.

File to upload: <input type="file" name="upfile" id="fileup"><br/>
<form method="POST" enctype="multipart/form-data" action="http://localhost:7071/api/HttpTrigger-Java">
  Notes about the file: <input type="text" name="note"><br/>
  <input type="hidden" name="file_base64" id="file_base64"><br/>
  <input type="submit" value="Press"> to upload the file!
</form>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js">/script>
<script>
$(document).ready(function(){
    $("#fileup").change(function(){
        var v = $(this).val();
        var reader = new FileReader();
        reader.readAsDataURL(this.files[0]);
        reader.onload = function(e){
            console.log(e.target.result);
            $('#file_base64').val(e.target.result);
        };
    });
});
</script>

The form above will post the header and body of base64 file chunk as below.

Header:
Content-Disposition: form-data; name="file_base64"
Body:
data:image/jpg;base64,iVBORw0KGgoAAAANSUhEUgAAB.............

My Java code in Azure Function:

import java.io.ByteArrayOutputStream;
import java.util.Base64;

if (header.equals("Content-Disposition: form-data; name=\"file_base64\"")) {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    multipartStream.readBodyData(baos);
    String content = baos.toString();
    // System.out.println(content);
    int index = content.indexOf(",")+1; // Get the index of base64 string in data-uploaded string
    byte[] imgBytes = Base64.getDecoder().decode(content.substring(index)); // convert image base64 string to image byte arrays
    ....
    // To upload image byte array to Blob Storage
    // You can get the upload image filename from the form input `note`, please notes the order of form input elements.
} else {
    multipartStream.readBodyData(System.out);
}

Question:

I have a HttpTrigger function with request body as

{
"id":"222",
"name":"some name"
}

I want to get the Id from request body into @BlobOuput path like below

@BlobOutput(name="blob",  path="input-blob/{body.id}")

Is there any way to achieve this functionality.


Answer:

Regarding how to read @BlobOuput path from httptrigger request body, please refer to the following steps.

I use the following request body for test

{
"BlobName":"test.txt",
 "Content":"test"
}
  1. Create a custom class as the request body.
public class BlobInfo {
    public String Content;
    public String BlobName;

    public String getContent() {
        return Content;
    }

    public void setContent(String content) {
        Content = content;
    }

    public String getBlobName() {
        return BlobName;
    }

    public void setBlobName(String blobName) {
        BlobName = blobName;
    }
}
  1. Function code
@FunctionName("HttpExample")
    public HttpResponseMessage run(
            @HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<BlobInfo>> request, // request body gets automatically de-serialized into the custom object you create
            @BlobOutput(name="output",  path = "test/{BlobName}" //  use the custom object's property you need as file name
                    ,connection = "AzureWebJobsStorage") OutputBinding<String> outputItem,
            final ExecutionContext context) throws IOException {
        context.getLogger().info("Java HTTP trigger processed a request.");
        BlobInfo body = request.getBody().get();
        context.getLogger().info(body.Content);
        context.getLogger().info(body.BlobName);
        outputItem.setValue("Hello World!");

        return request.createResponseBuilder(HttpStatus.OK).body("success").build();

    }
  1. Test
POST <function url>
Content-Type:application/json

{
 "BlobName":"test.txt",
 "Content":"test"
}

Question:

I am getting the above message when trying to run azure function sample(Microsoft Documentation)

I am getting this error when trying to run from eclispe. Dotnet cli is working from command prompt. Thanks.


Answer:

Followed this official document , you need to install .NET Core 2.0.

Then you need to install Azure Functions Core Tools 2.X via the command:

npm install -g azure-functions-core-tools@core

Hope it helps you.

Question:

I am trying to upload a file via a POST request. Using the HttpRequestMessage<File> gives me the error below. If I use HttpRequestMessage<byte[]> the code works but I am missing the meta-data like file name and other details which I need to upload as well.

Code:

@FunctionName("HttpExample")
    public String run(@HttpTrigger(name = "req", methods = { HttpMethod.GET,
            HttpMethod.POST }, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<File> request,
            final ExecutionContext context) { ... }

Error:

Executed 'Functions.HttpExample' (Failed, Id=7349f061-9213-4607-a757-8231c41078d2)
[27/02/2020 10:18:15] System.Private.CoreLib: Exception while executing function: Functions.HttpExample. System.Private.CoreLib: Result: Failure
[27/02/2020 10:18:15] Exception: ClassCastException: Cannot convert com.microsoft.azure.functions.worker.binding.RpcHttpRequestDataSource@74386ca9to type com.microsoft.azure.functions.HttpRequestMessage<java.io.File>
[27/02/2020 10:18:15] Stack: java.lang.ClassCastException: Cannot convert com.microsoft.azure.functions.worker.binding.RpcHttpRequestDataSource@74386ca9to type com.microsoft.azure.functions.HttpRequestMessage<java.io.File>
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.binding.DataOperations.generalAssignment(DataOperations.java:191)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.binding.DataOperations.apply(DataOperations.java:120)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.binding.DataSource.computeByType(DataSource.java:56)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.binding.RpcHttpRequestDataSource.computeByType(RpcHttpRequestDataSource.java:20)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.binding.DataSource.computeByName(DataSource.java:42)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.binding.RpcHttpRequestDataSource.computeByName(RpcHttpRequestDataSource.java:20)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.binding.BindingDataStore.getDataByName(BindingDataStore.java:55)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.broker.ParameterResolver.resolve(ParameterResolver.java:59)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.broker.ParameterResolver.resolve(ParameterResolver.java:42)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.broker.JavaMethodExecutor.execute(JavaMethodExecutor.java:52)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.broker.JavaFunctionBroker.invokeMethod(JavaFunctionBroker.java:53)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:33)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:10)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.handler.MessageHandler.handle(MessageHandler.java:45)
[27/02/2020 10:18:15]   at com.microsoft.azure.functions.worker.JavaWorkerClient$StreamingMessagePeer.lambda$onNext$0(JavaWorkerClient.java:92)
[27/02/2020 10:18:15]   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[27/02/2020 10:18:15]   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[27/02/2020 10:18:15]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[27/02/2020 10:18:15]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[27/02/2020 10:18:15]   at java.lang.Thread.run(Thread.java:748)
[27/02/2020 10:18:15] .

Answer:

It is impossible to directly use File. There is not implementation for converting your request to HttpRequestMessage<java.util.Optional<java.io.File>>.

A solution is to upload file with multipart/form-data. There is already a post about this, you may refer to: upload with multipart/form-data. There is also a known problem: Azure function will lose some unreadable bytes, to solve it, you may see the update in that answer.

Another easier solution, is to keep using byte array. However, at the same time, you may put the file name and other metadatas into query parameters. For example: ?filename=xxx&other=xxxx.

Then you can get file content from byte array, and all the metadatas from query parameters:

    @FunctionName("HttpExample")
    public HttpResponseMessage run(
            @HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<byte[]>> request,
            final ExecutionContext context) {
        context.getLogger().info("Java HTTP trigger processed a request.");

        byte[] body = request.getBody().get();

        Map<String, String> queryParameters = request.getQueryParameters();
        String fileName = queryParameters.get("fileName");

        return request.createResponseBuilder(HttpStatus.OK).body(fileName + " -> " + new String(body)).build();
    }

Question:

I'm trying to utilize the spring-azure-starter-storage plugin for Spring to manage BLOBS in Azure Storage.

I've followed the following guide on how to setup the storage account and the application, and it works all fine until i turn on "Secure transfer required" in Confuration for the storage account. (which is required for my project)

https://docs.microsoft.com/en-us/azure/java/spring-framework/configure-spring-boot-starter-java-app-with-azure-storage

my application.properties looks like this:

spring.cloud.azure.credential-file-path=my.azureauth
spring.cloud.azure.resource-group=xxxx
spring.cloud.azure.region=xxxxxx
spring.cloud.azure.storage.account=xxxx

The error i'm getting on upload is:

[http-nio-8080-exec-4] ERROR com.microsoft.azure.spring.cloud.storage.BlobStorageResource - Failed to open output stream of cloud blob
com.microsoft.azure.storage.StorageException: The account being accessed does not support http
....

Whitch makes sence. But how do i configure the connection to utilize https?

EDIT: I have since successfully implemented this using another approach shown in this tutorial: https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-java-legacy


Answer:

Summarize the answer to close this issue:

You should follow this article to configure https connection, a snippet as below:

public static final String storageConnectionString =
"DefaultEndpointsProtocol=https;" +
"AccountName=<account-name>;" +
"AccountKey=<account-key>";

Question:

I'm trying to access one drive for business oAuth v2.0 through java code. A/c to the specification given by one drive api authenticatoin : https://dev.onedrive.com/auth/aad_oauth.htm, we need to get authorization code after registering the application in azure ad in order to access onedrive for business.

So i have tried it in Advanced Rest Client (Chrome extension) and was successfully getting the auth code in the redirect uri(with response code 302 which is expected).

But, when i try to implement it in java using httpclient/URLConnection/httpURLConection/default httpURLConnection/jsoup, the response code which i get is 200 OK and i couldn't find the header field "Location" (in which we can get auth code along with redirect uri).

Is there any specific changes that should be made for response in java client to be as exact as in ARC??

Does java methods just discard these elements?


Answer:

Based on my understanding, you want to request an URL without redirect automatically like browser.

As I know, the Apache HttpClient will do the following redirect with the response code 30x if not set the ClientPNames.HANDLE_REDIRECTS='http.protocol.handle-redirects' with a boolean value, please see the doc http://hc.apache.org/httpcomponents-client-4.2.x/tutorial/html/httpagent.html#d5e1169.

ClientPNames.HANDLE_REDIRECTS='http.protocol.handle-redirects': defines whether redirects should be handled automatically. This parameter expects a value of type java.lang.Boolean. If this parameter is not set HttpClient will handle redirects automatically.

However, I don't think the URLConnection, HTTPURLConnection & HttpsURLConnection will automatically redirect, even jsoup implemented by them.

Here are some code shown how to prevent redirect automatically and extract the header.

1.Using HttpClient

HttpClient client = new DefaultHttpClient();
client.getParams().setParameter(ClientPNames.HANDLE_REDIRECTS, false);
HttpGet request = new HttpGet("<URL>");
HttpResponse response = client.execute(request);

//get all headers       
Header[] headers = response.getAllHeaders();
for (Header header : headers) {
    System.out.println("Key : " + header.getName() + " ,Value : " + header.getValue());
}

2.Using HttpURLConnection

URL url = new URL("<URL>");
URLConnection conn = url.openConnection();

//get all headers
Map<String, List<String>> map = conn.getHeaderFields();
for (Map.Entry<String, List<String>> entry : map.entrySet()) {
    System.out.println("Key : " + entry.getKey() + " ,Value : " + entry.getValue());
}

Hope it helps. Best Regards.