Hot questions for Using Ubuntu in scala

Question:

When running the Scala interpreter in Ubuntu 14.04, I get the following message printed as the first line:

Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar 

Followed by the familiar "Welcome to Scala" message.

I'm worried because I haven't seen that when running Scala before - what does it mean, is it dangerous, etc?

Apparently the environment variable $JAVA_TOOL_OPTIONS is set to -javaagent:/usr/share/java/jayatanaag.jar - I didn't set that, but what did and why? Can I safely unset it?

Additional info:

  • recently installed Android Studio
  • The word "ayatana" in the JAR's name might point to Ubuntu's project Ayatana

Answer:

You can disable jayatana just for the current shell session by unsetting JAVA_TOOL_OPTIONS like so:

unset JAVA_TOOL_OPTIONS

That way it will still be enabled for applications needing it.

Question:

I have a very simple scala jcuda program that adds a very large array. Everything compiles and runs just fine until I want to copy more than 4 bytes from my device to host. I am getting CUDA_ERROR_INVALID_VALUE when I try to copy more than 4 bytes.

// This does pukes and gives CUDA_ERROR_INVALID_VALUE
var hostOutput = new Array[Int](numElements)
cuMemcpyDtoH(
  Pointer.to(hostOutput),
  deviceOutput,
  8
)

// This runs just fine
var hostOutput = new Array[Int](numElements)
cuMemcpyDtoH(
  Pointer.to(hostOutput),
  deviceOutput,
  4
)

To give better context of the actual program bellow is my kernel code which compiles and runs just fine:

extern "C"
__global__ void add(int n, int *a, int *b, int *sum) {
    int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i<n)
    {
        sum[i] = a[i] + b[i];
    }
}

Also I then translated some java sample code into my scala code. Anyway bellow is the entire program that runs:

package dev

import jcuda.driver.JCudaDriver._

import jcuda._
import jcuda.driver._
import jcuda.runtime._

/**
 * Created by dev on 6/7/15.
 */
object TestCuda {
  def init = {
    JCudaDriver.setExceptionsEnabled(true)

    // Input vector

    // Output vector

    // Load module
    // Load the ptx file.

    val kernelPath = "/home/dev/IdeaProjects/jniopencl/src/main/resources/kernels/JCudaVectorAddKernel30.cubin"

    cuInit(0)

    val device = new CUdevice
    cuDeviceGet(device, 0)
    val context = new CUcontext
    cuCtxCreate(context, 0, device)

    // Create and load module
    val module = new CUmodule()
    cuModuleLoad(module, kernelPath)

    // Obtain a function pointer to the kernel function.
    var add = new CUfunction()
    cuModuleGetFunction(add, module, "add")

    val numElements = 100000

    val hostInputA = 1 to numElements toArray
    val hostInputB = 1 to numElements toArray
    val SI: Int = Sizeof.INT.asInstanceOf[Int]

    // Allocate the device input data, and copy
    // the host input data to the device
    var deviceInputA = new CUdeviceptr
    cuMemAlloc(deviceInputA, numElements * SI)
    cuMemcpyHtoD(
      deviceInputA,
      Pointer.to(hostInputA),
      numElements * SI
    )

    var deviceInputB = new CUdeviceptr
    cuMemAlloc(deviceInputB, numElements * SI)
    cuMemcpyHtoD(
      deviceInputB,
      Pointer.to(hostInputB),
      numElements * SI
    )

    // Allocate device output memory
    val deviceOutput = new CUdeviceptr()
    cuMemAlloc(deviceOutput, SI)

    // Set up the kernel parameters: A pointer to an array
    // of pointers which point to the actual values.
    val kernelParameters = Pointer.to(
      Pointer.to(Array[Int](numElements)),
      Pointer.to(deviceInputA),
      Pointer.to(deviceInputB),
      Pointer.to(deviceOutput)
    )

    // Call the kernel function
    val blockSizeX = 256
    val gridSizeX = Math.ceil(numElements / blockSizeX).asInstanceOf[Int]
    cuLaunchKernel(
      add,
      gridSizeX, 1, 1,
      blockSizeX, 1, 1,
      0, null,
      kernelParameters, null
    )

    cuCtxSynchronize

    // **** Code pukes here with that error
    // If I comment this out the program runs fine
    var hostOutput = new Array[Int](numElements)
    cuMemcpyDtoH(
      Pointer.to(hostOutput),
      deviceOutput,
      numElements
    )

    hostOutput.foreach(print(_))
  }
}

Anyway, just to let you know the specs of my computer. I'm running Ubuntu 14.04 on an optimus setup with a GTX 770M card which is compute 3.0 capable. I'm also running NVCC version 5.5. Lastly I'm running scala version 2.11.6 with Java 8. I'm a noob and would greatly appreciate any help.


Answer:

Here

val deviceOutput = new CUdeviceptr()
cuMemAlloc(deviceOutput, SI)

you are allocating SI bytes - which is 4 bytes, as the size of one int. Writing more than 4 bytes to this device pointer will mess up things. It should be

cuMemAlloc(deviceOutput, SI * numElements)

And similarly, I think that the call in question should be

cuMemcpyDtoH(
  Pointer.to(hostOutput),
  deviceOutput,
  numElements * SI
)

(note the * SI for the last parameter).

Question:

The project allows user to download a file which the downloaded file will be the zipped file. The downloaded zipped file works well except that the empty folder will not be included in the zipped folder. One of the developer implemented with the code below with his MacBook which doesn't trigger any error. However, when the source code is executed in ubuntu machine, error occurs.

The code:

def zipFolder(folderNamePath: String, subDirectory: String,zip: ZipOutputStream): Unit = {

    val file = new File(folderNamePath)
    val readBuffer = new Array[Byte](Buffer)
    val fileList = file.list()

    var i = 0
    for( i <- 0 until fileList.length ){
      val path = folderNamePath + "/" + fileList(i)
      val currFile = new File(path)
      if(currFile.isDirectory){
        val filePath = currFile.getPath
        zipFolder(filePath, subDirectory + '/' + fileList(i) ,zip)

      }else{
        val anEntry = new ZipEntry(subDirectory + '/' + fileList(i))
        val bis = new BufferedInputStream(new FileInputStream(path), Buffer)
        zip.putNextEntry(anEntry)
        var bytesIn: Int = -1
        while({
          bytesIn = bis.read(readBuffer, 0, Buffer)
          bytesIn != -1
        }){
          zip.write(readBuffer, 0, bytesIn);
        }

        bis.close()
      }
    }

    //THE CODE BELOW IS TO ZIP EMPTY DIRECTORY
    if(fileList.length == 0){
      val path = folderNamePath
      val anEntry = new ZipEntry(subDirectory)
      val bis = new BufferedInputStream(new FileInputStream(path + "/"), Buffer)
      zip.putNextEntry(anEntry)

      bis.close()
    }

  }

And the error logged:

! @75akh0a6a - Internal server error, for (POST) 
[/storage_ws/download_directory_file] ->

play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution 
exception[[FileNotFoundException: /home/jarvis/project/storage-api/media/jarvis/test/Test/Testing (Is a directory)]]
at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:280)
at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:206)
at play.api.GlobalSettings$class.onError(GlobalSettings.scala:160)
at play.api.DefaultGlobal$.onError(GlobalSettings.scala:188)
at play.api.http.GlobalSettingsHttpErrorHandler.onServerError(HttpErrorHandler.scala:98)
at play.filters.cors.AbstractCORSPolicy$$anonfun$2.applyOrElse(AbstractCORSPolicy.scala:146)
at play.filters.cors.AbstractCORSPolicy$$anonfun$2.applyOrElse(AbstractCORSPolicy.scala:145)
at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:346)
at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:345)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
Caused by: java.io.FileNotFoundException: /home/jarvis/project/storage-api/media/user/jarvis/Test/Testing (Is a directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at tools.ZipUtils$.zipFolder(ZipUtils.scala:122)
at tools.ZipUtils$$anonfun$zipFolder$1.apply$mcVI$sp(ZipUtils.scala:101)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
at tools.ZipUtils$.zipFolder(ZipUtils.scala:96)
at tools.IO$.handleDownloadDirectoryFile(IO.scala:176)
at controllers.FileHandleController$$anonfun$downloadDirectoryFile$1$$anonfun$apply$50.apply(FileHandleController.scala:586)

The code executed well without the following case if(fileList.length == 0){ for zipping empty folder, which the zipped folder will exclude the empty folder.


Answer:

The problem happens because you are trying to open a FileInputStream on a directory instead of a file.

Indeed, you see the exception message here:

Caused by: java.io.FileNotFoundException: /home/jarvis/project/storage-api/media/user/jarvis/Test/Testing (Is a directory)
at java.io.FileInputStream.open0(Native Method)

Here you are with a slightly modified version of your method that should solve the problem. Please note that I replaced all the '/' occurrences with File.separator instead.

def zipFolder(folderNamePath: String, subDirectory: String,zip: ZipOutputStream): Unit = {

    val file = new File(folderNamePath)
    val readBuffer = new Array[Byte](Buffer)
    val fileList = file.list()

    var i = 0
    for( i <- 0 until fileList.length ){
      val currFile = new File(folderNamePath, fileList(i))
      if(currFile.isDirectory){
        val filePath = currFile.getPath
        zipFolder(filePath, subDirectory + File.separator + fileList(i) ,zip)

      }else{
        val path = folderNamePath + File.separator + fileList(i)
        val anEntry = new ZipEntry(subDirectory + File.separator + fileList(i))
        val bis = new BufferedInputStream(new FileInputStream(path), Buffer)
        zip.putNextEntry(anEntry)
        var bytesIn: Int = -1
        while({
          bytesIn = bis.read(readBuffer, 0, Buffer)
          bytesIn != -1
        }){
          zip.write(readBuffer, 0, bytesIn);
        }

        bis.close()
      }
    }

    //THE CODE BELOW IS TO ZIP EMPTY DIRECTORY
    if(fileList.length == 0){
      zip.putNextEntry(new ZipEntry(subDirectory + File.separator))
      zip.closeEntry()
    }
}

The empty folder will only be visible when the zipped file is extracted. At least, for Ubuntu default zip tool(Archive Manage)

Question:

I am writing a java/scala program which generates pdf files for various different languages (like Chinese, Persian, Tamil). I am using itext.

The workflow is that I copy and paste some foreign language text into a UI I've built, click "Generate", then it creates and saves a nicely formatted pdf with that text inside.

My program is expandable in that more languages can be added (currently supporting about 8), but that requires adding more fonts, and manually downloading and testing new fonts is quite time consuming and doesn't scale well. Plus it's hard to know if a font supports all the characters (e.g. for Chinese) and I'm not going to visually notice if the odd character gets dropped out.

What would be nice is if my app could search my system for existing fonts which can render that text. This app is only going to be run on my computer so I'm not concerned about portability issues.

For example, libreoffice has no troubles rendering Chinese and Persian. Is it possible to find and reuse whatever fonts libreoffice uses? Or my browser? Libreoffice itself seems to use some kind of automatic font redirection for if I paste in some Chinese while the font is say Liberation Serif, it still manages to render the Chinese. I also prefer the Libreoffice fonts to the free ones I find on the internet. It's looks professional and I trust it more.

I haven't been able to find any good documentation explaining how Ubuntu and/or Libreoffice is magically handling this and I don't know of a good tool for searching my system for existing fonts that are able to render certain code points. Ubuntu has a Font Viewer program but it's not searchable by code point.

I am using 14.04 LTS and itext 5.0.6. Scala or Java code snippets are fine.

Thanks in advance!

-Rohan


Answer:

I ended up building a utility which recursively searches a directory of fonts for those which can render a piece of text. (The crucial method is font.canDisplay(Char))

On Ubuntu a good directory to target is /usr/share/fonts/truetype.

Of the fonts found, my utility would render the text with that font so that I could see which fonts I preferred the look of (and could then pick one through a UI).

To find all the fonts in a particular directory I used code something like this (I've removed UI code to make it more like a script):

val rootFontsDir = {
  val fontsDirPath = "/usr/share/fonts/truetype/" // Customise this
  val dir = new File(fontsDirPath)
  if (!dir.exists())
    throw new RuntimeException(s"Nothing found at path: '$fontsDirPath'")
  if (!dir.isDirectory)
    throw new RuntimeException(s"Object at path: '$fontsDirPath' isn't a directory")
  dir
}

def findFontFilesInDir(dir: File): Seq[File] = {
  val files = dir.listFiles()
  val subDirs = files.filter(_.isDirectory)
  val fontFiles = files.filter(_.isFile).filter(_.getName.endsWith(".ttf"))

  val subDirFontFiles = subDirs.flatMap(findFontFilesInDir)

  subDirFontFiles ++ fontFiles
}

case class FontInfo(font: Font, file: File)

val allFontInfo = findFontFilesInDir(rootFontsDir).map {
  case fontFile => FontInfo(Font.createFont(Font.TRUETYPE_FONT, fontFile), fontFile)
}

Once you have allFontInfo you can search for fonts that can render some text quite easily:

val sampleText = "尺度衡量人"

val matchingFontInfo = allFontInfo.filter(fontInfo => sampleText.forall(char => fontInfo.font.canDisplay(char)))

Then just loop over the results to see what the fonts are:

matchingFontInfo.foreach {
  case FontInfo(font, file) =>
    println(s"Text: '$sampleText' can be rendered by font '${font.getFontName}' found at '${file.getPath}'")
}

Or if you want to actually see it rendered so that you can compare visually:

matchingFontInfo.zipWithIndex.map {
  case (FontInfo(baseFont, file), index) =>
    val largerFont = baseFont.deriveFont(Font.PLAIN, 24)
    val label = new JLabel(s"[$index] ${baseFont.getFontName} : ${sampleText} (${file.getPath})")
    label.setFont(largerFont)
    // Add the label into a UI component ...
}

In my code the sample text was fed in through a UI rather than being hard coded as above and I spat out all the labels into a JFrame.