Hot questions for Using Azure in multithreading

Top Java Programmings / Azure / multithreading

Question:

I initialize a database instance in an asynctask class doInBackground(). The database instance and its methods are in a separate class (the one I will show) From the asynctask I start another thread in a class for my Azure client, which eventually queries the database class to populate my azure db client. Going back to my asynctask, I then access the db again (simultaneously as the first query) to query the db for separate information. It seems having the two cursors going simultaneously from separate threads querying is causing me problems.

Below are the methods in the db class which use the first cursor to send stuff to Azure. These methods are called from the separate thread run in an Azure class. The Azure class is called in the asynctask class. As you will see in the logcat, this works fine.

    public ArrayList<WeatherEvent> GetUnsychedWeatherEvent() {
    String query = COLUMN_SYNCHED + " = 0 ";
    Cursor cursor = db.query(TABLE_WEATHER, null, query, null, null, null, null);

    ArrayList<WeatherEvent> unsychedWeather = new ArrayList<>();
    while (cursor.moveToNext()) {
        unsychedWeather.add( GetWeatherEvent(cursor) );
    }
    cursor.close();

    return unsychedWeather;

}

//get weather event from db for azure
//TODO: finish populating this
public WeatherEvent GetWeatherEvent(Cursor cursor) {
    WeatherEvent wEvent = new WeatherEvent();
    wEvent.Latitude = cursor.getDouble(cursor.getColumnIndex(COLUMN_LATITUDE));
    wEvent.Longitude = cursor.getDouble(cursor.getColumnIndex(COLUMN_LONGITUDE));
    wEvent.CurrTemp = cursor.getString(cursor.getColumnIndex(COLUMN_CURRENT_TEMPERATURES));
    wEvent.CurrDesc = cursor.getString(cursor.getColumnIndex(COLUMN_CURRENT_WEATHER_DESCRIPTION));
    System.out.println("From DB, current temp: " + wEvent.CurrTemp);
    try {
        wEvent.Time = timeStampFormat.parseDateTime(cursor.getString(cursor.getColumnIndex(COLUMN_TIME_STAMP)));
    } catch (Exception e) {
        Log.d("GetEvent()", "Error parsing date " + e.toString());
    }

    return wEvent;
}

These next methods comes from the same db class. The 'dailycursor' here is what is causing me trouble. These methods are called in the asynctask class.

  public ArrayList<WeatherEvent> DailyWeatherEvents(ArrayList<WeatherEvent> dailyWeatherEvents) {
        Log.d("DB Access", "got to dailyweatherevents");

    //todo: more specific time query
    Cursor dailycursor = db.rawQuery("SELECT avg(" + COLUMN_CURRENT_TEMPERATURES + "), " + COLUMN_TIME_STAMP + ", " +
            "CASE WHEN  strftime('%M', " + COLUMN_TIME_STAMP + ") < '30' " +
            "THEN strftime('%H', " + COLUMN_TIME_STAMP + ") " +
            "ELSE strftime('%H', " + COLUMN_TIME_STAMP + ", '+1 hours') END " +
            "FROM " + TABLE_WEATHER +  " "+
            "GROUP BY strftime('%H', " + COLUMN_TIME_STAMP + ", '+30 minutes')", null);

    //ArrayList<WeatherEvent> dailyWeatherEvents = new ArrayList<>();
    try {
        if (dailycursor.moveToFirst()) {
            while (dailycursor.moveToNext()) {
                dailyWeatherEvents.add(GetDailyWeatherEvent(dailycursor));

            }
            dailycursor.close();
            System.out.println("Size of dailyinfo " + dailyWeatherEvents.size());
        }
        } catch (Exception e) {
        e.printStackTrace();
    }
    return dailyWeatherEvents;
}

public WeatherEvent GetDailyWeatherEvent(Cursor dailycursor) {
    WeatherEvent wEvent = new WeatherEvent();
   // wEvent.Latitude = dailycursor.getDouble(dailycursor.getColumnIndex(COLUMN_LATITUDE));
  //  wEvent.Longitude = dailycursor.getDouble(dailycursor.getColumnIndex(COLUMN_LONGITUDE));
    wEvent.CurrTemp = dailycursor.getString(dailycursor.getColumnIndex(COLUMN_CURRENT_TEMPERATURES));
    System.out.println(wEvent.CurrTemp + " mariah this comes from cursor");
    wEvent.CurrDesc = dailycursor.getString(dailycursor.getColumnIndex(COLUMN_CURRENT_WEATHER_DESCRIPTION));
    try {
        wEvent.Time = timeStampFormat.parseDateTime(dailycursor.getString(dailycursor.getColumnIndex(COLUMN_TIME_STAMP)));
    } catch (Exception e) {
        Log.d("GetEvent()", "Error parsing date " + e.toString());
    }
    wEvent.CurrIcon = dailycursor.getString(dailycursor.getColumnIndex(COLUMN_CURRENT_ICONS));
    wEvent.SevereWeatherPresent = dailycursor.getInt(dailycursor.getColumnIndex(COLUMN_SEVERE_WEATHER_PRESENT))>0;

    return wEvent;

}

The lines from the asynctask class that start all of this. The WeatherDBAccess is the DB class. The WeatherAzureAccess is the azure class:

  WeatherDBAccess._context = this.context;
            WeatherDBAccess.Instance().AddWeatherEvent(wEvent);
            //starts azure
            WeatherAzureAccess.context = this.context;
            WeatherAzureAccess.Instance();


        //prep dailyinfo arraylist

            dailyinfo = WeatherDBAccess.Instance().DailyWeatherEvents(dailyinfo);

And this is my logcat, the "From DB, current temp: XX" lines are printing from the methods for azure:

    01-12 15:15:43.869 29979-30189/pdgt.cat.com.noaahso I/SQLiteAssetHelper: successfully opened database HSO.sqlite
01-12 15:15:43.869 29979-30189/pdgt.cat.com.noaahso D/WeatherDBAccess: Instance Created /data/data/pdgt.cat.com.noaahso/databases/HSO.sqlite
01-12 15:15:43.869 29979-30189/pdgt.cat.com.noaahso I/System.out: Current Temp: 54
01-12 15:15:43.899 29979-30189/pdgt.cat.com.noaahso I/System.out: Weather Event Added
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso D/DB Access: got to dailyweatherevents
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso E/CursorWindow: Failed to read row 1, column -1 from a CursorWindow which has 3 rows, 3 columns.
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err: java.lang.IllegalStateException: Couldn't read row 1, col -1 from CursorWindow.  Make sure the Cursor is initialized correctly before accessing data from it.
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso D/AzureAccess: WeatherSync thread STARTED!
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at android.database.CursorWindow.nativeGetString(Native Method)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at android.database.CursorWindow.getString(CursorWindow.java:451)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at android.database.AbstractWindowedCursor.getString(AbstractWindowedCursor.java:51)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at pdgt.cat.com.noaahso.WeatherDBAccess.GetDailyWeatherEvent(WeatherDBAccess.java:235)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at pdgt.cat.com.noaahso.WeatherDBAccess.DailyWeatherEvents(WeatherDBAccess.java:219)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at pdgt.cat.com.noaahso.WeatherTask.doInBackground(WeatherTask.java:228)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at pdgt.cat.com.noaahso.WeatherTask.doInBackground(WeatherTask.java:47)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at android.os.AsyncTask$2.call(AsyncTask.java:288)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at java.util.concurrent.FutureTask.run(FutureTask.java:237)
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 49
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
01-12 15:15:43.919 29979-30189/pdgt.cat.com.noaahso W/System.err:     at java.lang.Thread.run(Thread.java:818)
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 49
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 49
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 49
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 50
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 50
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 50
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 51
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 51
01-12 15:15:43.919 29979-29979/pdgt.cat.com.noaahso D/AsyncTask: got to onpostExecute
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 51
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 51
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 51
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 51
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 51
01-12 15:15:43.919 29979-30298/pdgt.cat.com.noaahso I/System.out: From DB, current temp: 51

Do I need to use a FutureCallback on the first cursor to know when to start the second cursor? Or do I need to make my first cursor global and wait until the first query is done before I reset the one cursor to do my second query? Looking for a solution I have learned that starting the second cursor is expensive. My problem is that the second cursor would be used rarely (possibly only onCreate()) and the first cursor will almost be always going as it's constantly used to send stuff to Azure. Also FYI I close my db onDestroy().


Answer:

Firstly, your error is:

java.lang.IllegalStateException: Couldn't read row 1, col -1 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.

That means, that you have results (row 1), but you are trying to read an invalid column index (col -1).

The problem area is likely here:

wEvent.CurrTemp = dailycursor.getString(dailycursor.getColumnIndex(COLUMN_CURRENT_TEMPERATURES));

You are getting a -1 result from getColumnIndex because COLUMN_CURRENT_TEMPERATURES does not exist in your SQL query.

Cursor dailycursor = db.rawQuery("SELECT avg(" + COLUMN_CURRENT_TEMPERATURES + "), " + COLUMN_TIME_STAMP + ", " +
            "CASE WHEN  strftime('%M', " + COLUMN_TIME_STAMP + ") < '30' " +
            "THEN strftime('%H', " + COLUMN_TIME_STAMP + ") " +
            "ELSE strftime('%H', " + COLUMN_TIME_STAMP + ", '+1 hours') END " +
            "FROM " + TABLE_WEATHER +  " "+
            "GROUP BY strftime('%H', " + COLUMN_TIME_STAMP + ", '+30 minutes')", null);

You need to include all columns that you intent to read from in your SELECT statement:

Cursor dailycursor = db.rawQuery("SELECT avg(" + COLUMN_CURRENT_TEMPERATURES + "), " + COLUMN_CURRENT_TEMPERATURES + ", " + COLUMN_CURRENT_WEATHER_DESCRIPTION + ", " + COLUMN_CURRENT_ICONS + ", " + COLUMN_SEVERE_WEATHER_PRESENT + ", " + COLUMN_TIME_STAMP + ", " +
            "CASE WHEN  strftime('%M', " + COLUMN_TIME_STAMP + ") < '30' " +
            "THEN strftime('%H', " + COLUMN_TIME_STAMP + ") " +
            "ELSE strftime('%H', " + COLUMN_TIME_STAMP + ", '+1 hours') END " +
            "FROM " + TABLE_WEATHER +  " "+
            "GROUP BY strftime('%H', " + COLUMN_TIME_STAMP + ", '+30 minutes')", null);

Question:

I have high volume multi threaded java based application which needs to call REST based endpoints running on Microsoft cloud with Authorization token in the header retrieved using "Azure ADAL AcquireToken". I am using "AzureAD/azure-activedirectory-library-for-java" (Code sample below). Questions I have are -

  1. Do I need to make a call to retrieve the token using acquireToken method for each and every REST call I am going to make? If yes, then I believe the token I am going to get with latest call may change, then in that case are the requests if I have requests already made with previously retrieved tokens going to fail or Azure ADAL will still going to honor all the previously generated tokens?
  2. If token retrieved previously are not expected to be honored by Azure ADAL then what options I have to manage single token at a time and then making sure at a time only one token is used by all the requests? Do I need to implement some kind of single threaded cache to retrieve a token, maintain that token until it expires, make a call to get new token if expired and make all my multi threaded requests go through this single threaded cache to get the latest token? Any suggestions on this. If this is the case then its seems like a huge bottleneck in high volume multi threaded multi jvm application as far as scalability goes.

My code below. When I called the acquireToken method from a loop inside the main method, I mostly got 3 different types of token in 10 calls and all 3 different tokens seemed to work, but not sure if that how it should be called in a multi threaded applications.

package com.mycompany.msft.auth;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

import com.microsoft.aad.adal4j.AuthenticationContext;
import com.microsoft.aad.adal4j.AuthenticationResult;
import com.microsoft.aad.adal4j.ClientCredential;

public class ApplicationAuthExample {
    private final static String AUTHORIZATION_ENDPOINT = "https://login.microsoftonline.com/";
    private final static String ARM_ENDPOINT = "https://myendpoint";


    private static String credential = "my credential";
    private static String clientId = "my client id";
    private static String tenantId = "my tenant id";

    private static String url = AUTHORIZATION_ENDPOINT + tenantId ;

    AuthenticationContext context = null;
    AuthenticationResult result = null;
    ExecutorService service = null;

    public  AuthenticationResult getAuthToken() {
        try {
            service = Executors.newFixedThreadPool(1);


            context = new AuthenticationContext(url, false, service);

            Future<AuthenticationResult> future = null;


                ClientCredential cred = new ClientCredential(clientId, credential);
                future = context.acquireToken(ARM_ENDPOINT, cred, null);


            result = future.get();
        } catch (Exception ex) {
            System.out.println("Exception occurred:");
            ex.printStackTrace();
            System.exit(1);
        } finally {
            service.shutdown();
        }
        return result;
    }

    public static void main(String[] args) throws Exception {

        ApplicationAuthExample auth = new ApplicationAuthExample();

        for (int i =0 ; i< 10 ; i++) {
            AuthenticationResult result = auth.getAuthToken();
            // use adal to Authenticate

            System.out.println (i+ " Authorization" + "Bearer " + result.getAccessToken());
            System.out.println (i + " getExpiresOn" + result.getExpiresOn());

            //This token comes different in different calls. Which one should I use and which one not. 
            System.out.println (i+ " getExpiresOn" + result.getRefreshToken());


            System.out.println (i+" getExpiresOn" + result.getUserInfo());
            }

    }
}

Answer:

Per my experience, I think the key of the issue is the expiration time of the token. You can use the token as your wish after it expire. You can refer to the section Claims in id_tokens of https://azure.microsoft.com/en-us/documentation/articles/active-directory-v2-tokens/ to know that the token claim the time include Issued At, Expriation Time & `Not Before.

So you will need to use adal4j to acquire the token and request a refresh token when the previous one expires.

The lifetime of a default security token for a claims-based authentication deployment using AD FS 2.0 is 60 minutes.

If you want to increase the token expiration time of Azure AD, you can try to refer to the doc https://technet.microsoft.com/en-us/library/gg188586.aspx to configure the relying party token lifetime.

Question:

Just like the .NET SDK for Azure storage, the Java version has a CloudTable class used for executing operations against a specific Azure Table.

And while it seems that the .NET implementation isn't thread safe, I can't find any definitive answer for Java. So is the Java version thread safe or not?


Answer:

I'm thinking you've already seen the response to the issue you created on github https://github.com/Azure/azure-storage-java/issues/38. Re-posting the answer for those who come across this question in the future.

We do not make any thread-safety guarantees about our libraries. We also do not test them for thread-safety. Methods that are currently thread-safe may be thread-unsafe in future versions. Thank you for pointing this out to us. We will look into updating our documentation.

Thanks