Hot questions for Using Joda-Time in datetime parsing

Top Java Programmings / Joda-Time / datetime parsing

Question:

I'm parsing a number of news feeds and each item's pubDate follows the same format:

Sun, 11 Jun 2017 18:18:23 +0000

Unfortunately one feed does not:

Sat, 10 Jun 2017 12:49:45 EST

I have tried to parse the date with no luck using androids java date and SimpleDateFormat:

try {
    Calendar cal = Calendar.getInstance();
    TimeZone tz = cal.getTimeZone();
    SimpleDateFormat readDate = new SimpleDateFormat("EEE, d MMM yyyy HH:mm:ss Z");
    readDate.setTimeZone(TimeZone.getTimeZone("UTC"));
    Date date = readDate.parse(rssDateTime);
    SimpleDateFormat writeDate = new SimpleDateFormat("EEE, d MMM yyyy HH:mm:ss Z");
    writeDate.setTimeZone(tz);
    parsedDate = writeDate.format(date);
} catch (ParseException e) {
    e.printStackTrace();
}

Which throws and error:

java.text.ParseException: Unparseable date: "Sat, 3 Jun 2017 19:53:09 EST" (at offset 26)

I've also tried to do this using joda time:

DateTime dtUTC = null;
DateTimeZone timezone = DateTimeZone.getDefault();
DateTimeFormatter formatDT = DateTimeFormat.forPattern("EEE, d MMM yyyy HH:mm:ss Z");
DateTime dtRssDateTime = formatDT.parseDateTime(rssDateTime);
DateTime now = new DateTime();
DateTime nowUTC = new LocalDateTime(now).toDateTime(DateTimeZone.UTC);

long instant = now.getMillis();
long instantUTC = nowUTC.getMillis();
long offset = instantUTC - instant;
dtUTC = dtRssDateTime.withZoneRetainFields(timezone);
dtUTC = dtUTC.minusMillis((int) offset);
String returnTimeDate = "";
returnTimeDate = dtUTC.toString(formatDT);

Which throws an error:

Caused by: java.lang.IllegalArgumentException: Invalid format: "Sat, 10 Jun 2017 12:49:45 EST" is malformed at " EST"

Has anyone encountered this before?


Answer:

First of all, if you're starting a new project, I suggest you to use the new date-time API instead of joda-time (more on that below). Anyway, here's a solution for both.


Joda Time

The problem is that the pattern Z is the offset (in formats like +0000 or -0100), but the string EST is the timezone short name, which is parsed by the pattern z (take a look at jodatime javadoc for more details).

So, you need a pattern with optional sections, that can receive one or another at the same time. You can do that with the org.joda.time.format.DateTimeFormatterBuilder class.

First you need to create 2 instances of org.joda.time.format.DateTimeParser (one for Z, and other for z), and add them as optional parsers. Then you create the org.joda.time.format.DateTimeFormatter using the code below. Note that I also used java.util.Locale, just to make sure it parses the weekday and month names correctly (so you don't depend upon default locale, which can vary on each system/machine):

// offset parser (for "+0000")
DateTimeParser offsetParser = new DateTimeFormatterBuilder().appendPattern("Z").toParser();
// timezone name parser (for "EST")
DateTimeParser zoneNameParser = new DateTimeFormatterBuilder().appendPattern("z").toParser();
// formatter for both patterns
DateTimeFormatter fmt = new DateTimeFormatterBuilder()
    // append common pattern
    .appendPattern("EEE, d MMM yyyy HH:mm:ss ")
    // optional offset
    .appendOptional(offsetParser)
    // optional timezone name
    .appendOptional(zoneNameParser)
    // create formatter (use English Locale to make sure it parses weekdays and month names independent of JVM config)
    .toFormatter().withLocale(Locale.ENGLISH)
    // make sure the offset "+0000" is parsed
    .withOffsetParsed();

// parse the strings
DateTime est = fmt.parseDateTime("Sat, 10 Jun 2017 12:49:45 EST");
DateTime utc = fmt.parseDateTime("Sun, 11 Jun 2017 18:18:23 +0000");
System.out.println(est);
System.out.println(utc);

The output will be:

2017-06-10T12:49:45.000-04:00 2017-06-11T18:18:23.000Z

If they're not exactly like you were expecting (or you're still getting errors), please see the notes below.


Notes:

  • Note that EST was printed as a date/time with offset -0400. That's because EST internally became America/New_York timezone, which is now in Daylight Saving Time and its offset is -0400 (I could figure this out by doing DateTimeZone.forTimeZone(TimeZone.getTimeZone("EST")). The problem is: these 3-letter names are ambiguous and not standard, and joda-time just assumes a "default" for them. So, if you were not expecting this timezone, and you don't want to rely on defaults, you can use a map with custom values, like this:

    // mapping EST to some other timezone (I know it's wrong and Chicago is not EST, it's just an example)
    Map<String, DateTimeZone> map = new LinkedHashMap<>();
    map.put("EST", DateTimeZone.forID("America/Chicago"));
    // parser for my custom map
    DateTimeParser customTimeZoneParser = new DateTimeFormatterBuilder().appendTimeZoneShortName(map).toParser();
    DateTimeFormatter fmt = new DateTimeFormatterBuilder()
        // append common pattern
        .appendPattern("EEE, d MMM yyyy HH:mm:ss ")
        // optional offset
        .appendOptional(offsetParser)
        // optional custom timezone name
        .appendOptional(customTimeZoneParser)
        // optional timezone name (accepts all others that are not in the map)
        .appendOptional(zoneNameParser)
        // create formatter (use English Locale to make sure it parses weekdays and month names independent of JVM config)
        .toFormatter().withLocale(Locale.ENGLISH)
        // make sure the offset "+0000" is parsed
        .withOffsetParsed();
    System.out.println(fmt.parseDateTime("Sat, 10 Jun 2017 12:49:45 EST"));
    

This will parse EST as America/Chicago (I know it's wrong and Chicago is not EST, it's just an example of how you can change the defaults using a map), and the output will be:

2017-06-10T12:49:45.000-05:00

If you got an error with the first code above, you can also use this, mapping EST to the desired timezone (depending on the version of jodatime and Java you're using, EST might not be mapped to a default value and throws an exception, so using a custom map avoids this).


New Date-time API

As told in @Ole V.V.'s comment (and I didn't have time to write yesterday), joda-time is being replaced by the new Java's Date and Time API, which is far superior compared to the old Date and SimpleDateFormat classes.

If you're using Java >= 8, the java.time package is already part of the JDK. For Java <= 7 there's the ThreeTen Backport. And for Android, there's the ThreeTenABP (more on how to use it here).

If you're starting a new project, please consider the new API instead of joda-time, because in joda's website it says: Note that Joda-Time is considered to be a largely "finished" project. No major enhancements are planned. If using Java SE 8, please migrate to java.time (JSR-310).

The code below works for both. The only difference is the package names (in Java 8 is java.time and in ThreeTen Backport (or Android's ThreeTenABP) is org.threeten.bp), but the classes and methods names are the same.

The idea is very similar to jodatime, with minor differences:

  • you can use the optional section delimiters []
  • a set with custom timezone names (to map EST to some valid non-ambiguous timezone) is required (as EST is not mapped to any default)
  • a new class is used: ZonedDateTime, which represents a date and time with a timezone (so it covers both of your cases)

Just reminding that these classes are in java.time package (or in org.threeten.bp depending on what Java version you're using, as explained above):

// set with custom timezone names
Set<ZoneId> set = new HashSet<>();
// when parsing, ambiguous EST uses to New York
set.add(ZoneId.of("America/New_York"));

DateTimeFormatter fmt = new DateTimeFormatterBuilder()
    // append pattern, with optional offset (delimited by [])
    .appendPattern("EEE, d MMM yyyy HH:mm:ss[ Z]")
    // append optional timezone name with custom set for EST
    .optionalStart().appendLiteral(" ").appendZoneText(TextStyle.SHORT, set).optionalEnd()
    // create formatter using English locale to make sure it parses weekdays and month names correctly
    .toFormatter(Locale.ENGLISH);

ZonedDateTime est = ZonedDateTime.parse("Sat, 10 Jun 2017 12:49:45 EST", fmt);
ZonedDateTime utc = ZonedDateTime.parse("Sun, 11 Jun 2017 18:18:23 +0000", fmt);
System.out.println(est); // 2017-06-10T12:49:45-04:00[America/New_York]
System.out.println(utc); // 2017-06-11T18:18:23Z

The output will be:

2017-06-10T12:49:45-04:00[America/New_York] 2017-06-11T18:18:23Z

Note that in the first case, EST was set to America/New_York (as configured by the custom set). The appendZoneText does the trick, using the values in the custom set to resolve ambiguous cases.

And the second case was set to UTC, as the offset is +0000.

If you want to convert the first object to UTC, it's straighforward:

System.out.println(est.withZoneSameInstant(ZoneOffset.UTC)); // 2017-06-10T16:49:45Z

The output will be the New York's date/time converted to UTC:

2017-06-10T16:49:45Z

Instead of ZoneOffset.UTC, of course you can use any timezone or offset you want (using ZoneId and ZoneOffset classes, check the javadoc for more details).

Question:

Now I have two different formats of date written in string:

String date1 = "2018-10-12 18:01:01";// yyyy-MM-dd HH:mm:ss
String date2 = "2018-10-12 18:01";//yyyy-MM-dd HH:mm

I am using joda and I want to convert the string to DateTime,the basic way is to use two formatter to parse each of them:

DateTimeFormatter formatter1 = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss");
DateTimeFormatter formatter2 = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm");
DateTime dt1 = formatter1.parseDateTime(date1);
DateTime dt2 = formatter2.parseDateTime(date2);

Above code blocks works fine but it created two formatter,since the date formate is very similar(the latter one just lack of seconds),I am wonder if there is a way that I can just use one formatter to parse all of them or I have to use two formatter?

Note: due to the production enviroment limit,I can not use java8 now,so I want to the answer based on joda

Thanks in advance!


I just tried as below,and got IllegalArgumentException´╝Ü Invalid format

DateTimeFormatter formatter = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss");
DateTime dt1 = formatter.parseDateTime(date1);
DateTime dt2 = formatter.parseDateTime(date2);

Answer:

You can indicate that some parts of the format are optional using []

DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm[:ss]");
LocalDateTime dateTime = LocalDateTime.parse("2018-10-12 18:01:01", formatter);
LocalDateTime dateTime1 = LocalDateTime.parse("2018-10-12 18:01", formatter);
System.out.println(dateTime + " " + dateTime1);

result is

2018-10-12T18:01:01 2018-10-12T18:01

Please see Patterns for Formatting and Parsing section for more info.

Question:

I'm trying to convert into HH:MM format in Java

Here is my code

String date = "18:30:00.000Z";

    try {
        ZonedDateTime parsed = ZonedDateTime.parse(date);

        ZonedDateTime z = parsed.withZoneSameInstant(ZoneId.systemDefault());

        DateTimeFormatter fmt = DateTimeFormatter.ofPattern("hh:mm a", Locale.ENGLISH);
        System.out.println(fmt.format(z));
    } catch(Exception e) {
        e.printStackTrace();
    }

Is there anything wrong in my code

Here is the exception

java.time.format.DateTimeParseException: Text '18:30:00.000Z' could not be parsed at index 0

Answer:

The string you are trying to parse is not a date and time, but only a time. So the type ZonedDateTime is not the appropriate type to use here. Use OffsetTime instead, which is a class for holding just a time (without a date).

String input = "18:30:00.000Z";
OffsetTime time = OffsetTime.parse(input);
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("h:mm a");
String result = formatter.format(time);
System.out.println(result);

Output:

6:30 PM

Question:

I've configured formatter:

public static final DateTimeFormatter DATE_FORMATTER = new DateTimeFormatterBuilder()
        .append(forPattern("yyyy-MM-dd"))
        .append(forPattern("MM/dd/yy"))
        .append(forPattern("MMM dd, yyyy"))
        .toFormatter();

and trying to parse a string 2017-08-29

LocalDate.parse(dt, DATE_FORMATTER).toDateTimeAtStartOfDay().toLocalDateTime()

I'm getting error:

IllegalArgumentException: Invalid format: "2017-08-29" is too short

Error gone if I leave "yyyy-MM-dd" the only format in the builder.

Am I misusing the API? I would like the parser to try another format if it fails with first one.


Answer:

When you use the append method, you're creating a formatter that accepts all the three patterns, one after the another (all the three are required).

If you want to accept any of the three formats (just one of them), you must use appendOptional instead:

DateTimeFormatter DATE_FORMATTER = new DateTimeFormatterBuilder()
    .appendOptional(DateTimeFormat.forPattern("yyyy-MM-dd").getParser())
    .appendOptional(DateTimeFormat.forPattern("MM/dd/yy").getParser())
    .appendOptional(DateTimeFormat.forPattern("MMM dd, yyyy").getParser())
    .toFormatter();

Now you can parse any of the three formats:

System.out.println(LocalDate.parse("2017-08-29", DATE_FORMATTER).toDateTimeAtStartOfDay().toLocalDateTime());
System.out.println(LocalDate.parse("08/29/17", DATE_FORMATTER).toDateTimeAtStartOfDay().toLocalDateTime());
System.out.println(LocalDate.parse("Aug 29, 2017", DATE_FORMATTER).toDateTimeAtStartOfDay().toLocalDateTime());

All of the above outputs:

2017-08-29T00:00:00.000


Just one note: the third formatter uses the month short name (MMM), and the code above assumes that the system's default locale is English (when you create a formatter, by default it uses the language that corresponds to the system's default locale).

But this can be changed without notice, even at runtime, so it's better to specify a java.util.Locale in your formatter.

Example: if month name is always in English, just use the equivalent locale:

DateTimeFormatter DATE_FORMATTER = new DateTimeFormatterBuilder()
    .appendOptional(DateTimeFormat.forPattern("yyyy-MM-dd").getParser())
    .appendOptional(DateTimeFormat.forPattern("MM/dd/yy").getParser())
    .appendOptional(DateTimeFormat.forPattern("MMM dd, yyyy").getParser())
    // use English locale
    .toFormatter().withLocale(Locale.ENGLISH);

Just change the locale to the one that best fits your needs. Check the javadoc for more details.


As reminded in the comments, you can also create an array of parsers and use in the DateTimeFormatterBuilder:

// array with all possible patterns
DateTimeParser[] parsers = new DateTimeParser[] {
    DateTimeFormat.forPattern("yyyy-MM-dd").getParser(),
    DateTimeFormat.forPattern("MM/dd/yy").getParser(),
    DateTimeFormat.forPattern("MMM dd, yyyy").getParser() };

DateTimeFormatter DATE_FORMATTER = new DateTimeFormatterBuilder()
    // use array of all possible parsers
    .append(null, parsers)
    // use English locale
    .toFormatter().withLocale(Locale.ENGLISH);

This works the same way as the previous one.


Java new Date/Time API

Joda-Time is in maintainance mode and is being replaced by the new APIs, so I don't recommend start a new project with it. Even in joda's website it says: "Note that Joda-Time is considered to be a largely "finished" project. No major enhancements are planned. If using Java SE 8, please migrate to java.time (JSR-310).".

If you can't (or don't want to) migrate from Joda-Time to the new API, you can ignore this section.

If you're using Java 8, consider using the new java.time API. It's easier, less bugged and less error-prone than the old APIs.

If you're using Java <= 7, you can use the ThreeTen Backport, a great backport for Java 8's new date/time classes. And for Android, you'll also need the ThreeTenABP (more on how to use it here).

The code below works for both. The only difference is the package names (in Java 8 is java.time and in ThreeTen Backport (or Android's ThreeTenABP) is org.threeten.bp), but the classes and methods names are the same.

The API is very similar when it comes to create the formatter and parsing it:

DateTimeFormatter DATE_FORMATTER = new DateTimeFormatterBuilder()
    .appendOptional(DateTimeFormatter.ofPattern("yyyy-MM-dd"))
    .appendOptional(DateTimeFormatter.ofPattern("MM/dd/yy"))
    .appendOptional(DateTimeFormatter.ofPattern("MMM dd, yyyy"))
    // use English locale
    .toFormatter(Locale.ENGLISH);

System.out.println(LocalDate.parse("2017-08-29", DATE_FORMATTER).atStartOfDay());
System.out.println(LocalDate.parse("08/29/17", DATE_FORMATTER).atStartOfDay());
System.out.println(LocalDate.parse("Aug 29, 2017", DATE_FORMATTER).atStartOfDay());

All the above create a LocalDateTime with the value that corresponds to 2017-08-29T00:00.

You can also use optional patterns (delimited by []):

DateTimeFormatter DATE_FORMATTER = DateTimeFormatter.ofPattern("[yyyy-MM-dd][MM/dd/yy][MMM dd, yyyy]", Locale.ENGLISH);

This works the same way as the above.

Question:

I am trying to write a DateTimeFormatter that will allow me to take in multiple different string formats, and then convert the string formats to a specific type. Due to the scope of the project and the code that already exists, I cannot use a different type of formatter.

eg. I want to accept MM/dd/yyyy as well as yyyy-MM-dd'T'HH:mm:ss but then convert both to MM/dd/yyyy.

Could someone suggest ideas on how to do this with the org.joda.time.format?

I have not found a good/working example of this online.


Answer:

I'm using Joda-Time 2.9.7 and JDK 1.7.0_79.

You can use DateTimeFormatterBuilder.append method: it receives a printer (with a pattern used to print date/times) and an array of parsers with all possible input patterns:

import org.joda.time.DateTime;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
import org.joda.time.format.DateTimeFormatterBuilder;
import org.joda.time.format.DateTimeParser;

// MM/dd/yyyy format
DateTimeFormatter monthDayYear = DateTimeFormat.forPattern("MM/dd/yyyy");
// array of parsers, with all possible input patterns
DateTimeParser[] parsers = {
    // parser for MM/dd/yyyy format
    monthDayYear.getParser(),
    // parser for yyyy-MM-dd'T'HH:mm:ss format
    DateTimeFormat.forPattern("yyyy-MM-dd'T'HH:mm:ss").getParser() };
DateTimeFormatter parser = new DateTimeFormatterBuilder()
    // use the monthDayYear formatter for output (monthDayYear.getPrinter()) and parsers array for input (parsers)
    .append(monthDayYear.getPrinter(), parsers)
    // create formatter (using UTC to avoid DST problems)
    .toFormatter().withZone(DateTimeZone.UTC);

// test with MM/dd/yyyy
DateTime datetime1 = parser.parseDateTime("06/14/2017");
System.out.println(parser.print(datetime1)); // 06/14/2017

// test with yyyy-MM-dd'T'HH:mm:ss
DateTime datetime2 = parser.parseDateTime("2017-06-14T10:30:40");
System.out.println(parser.print(datetime2)); // 06/14/2017

I've used DateTimeZone.UTC to avoid problems with Daylight Saving Time.

For example, in my default timezone (America/Sao_Paulo), last year (2016), DST started at October 16th: at midnight, the clocks move 1 hour ahead (so, tecnically, midnight doesn't exist at this day, because time changes from 23:59:59 to 01:00:00).

The problem is, when parsing MM/dd/yyyy format, there's no fields for hour, minute or second, and the parser sets 0 as a default value for all these fields (so the hour becomes midnight). But if I try to parse in a date where DST starts (like 10/16/2016) and don't use the UTC as above, the code throws an exception because midnight doesn't exist in that day (due to DST hour shift).

Using UTC avoids this error, as DateTimeZone.UTC has no DST effects. With this, the code works independent of your system's default timezone.

The output is:

06/14/2017 06/14/2017


PS: As you care only about the date part (day/month/year), you could also use the org.joda.time.LocalDate class. To use it, just change the last part of the code (you can use the same parser):

// test with MM/dd/yyyy
LocalDate dt1 = parser.parseLocalDate("06/14/2017");
System.out.println(parser.print(dt1)); // 06/14/2017

// test with yyyy-MM-dd'T'HH:mm:ss
LocalDate dt2 = parser.parseLocalDate("2017-06-14T10:30:40");
System.out.println(parser.print(dt2)); // 06/14/2017

The output is the same:

06/14/2017 06/14/2017

Using a LocalDate is another way to avoid problems with Daylight Saving Time (as explained above). In this case, you don't need to set UTC, because LocalDate has no timezone information.