Thursday, June 2, 2016

Simple and useful Zip and unZip text

Simple and usefull textual zip and unzip.
First we use binary zip . but if we want to send it as a text message (e.g some message bus supports only text messages) we need to convert it to text by encoding to base 64 and vice versa to unzip it.
import java.io.ByteArrayInputStream
import java.nio.charset.StandardCharsets
import java.util.zip.{GZIPOutputStream, GZIPInputStream}
import org.apache.commons.codec.binary.Base64
import org.apache.commons.io.output.ByteArrayOutputStream

import scala.util.Try
  val zipToBinary: String => Array[Byte] = {txt =>
    val arrOutputStream = new ByteArrayOutputStream()
    val zipOutputStream = new GZIPOutputStream(arrOutputStream)
    zipOutputStream.write(txt.getBytes(StandardCharsets.UTF_8))
    zipOutputStream.close()
    arrOutputStream.toByteArray
  }


  val byteArrayToTxt : Array[Byte] => String = bytes => Base64.encodeBase64String(bytes)

  val txtToBinary : String => Array[Byte]= txt => Base64.decodeBase64(txt.getBytes(StandardCharsets.UTF_8))

  val binaryToTxt : Array[Byte] => String = bytes =>
    scala.io.Source.fromInputStream(new GZIPInputStream(new ByteArrayInputStream(bytes)))(StandardCharsets.UTF_8).mkString

  val zip:String => String = zipToBinary andThen byteArrayToTxt

  val unzip :String => String = txtToBinary andThen binaryToTxt


Usage :
val someLongTxt = "One morning, when Gregor Samsa woke from troubled dreams, he found himself transformed in his bed into a horrible vermin. He lay on his armour-like back, and if he lifted his head a little he could see his brown belly, slightly domed and divided by arches into stiff sections. The bedding was hardly able to cover it and seemed ready to slide off any moment. His many legs, pitifully thin compared with the size of the rest of him, waved about helplessly as he looked. \"What's happened to me?\" he thought. It wasn't a dream. His room, a proper human room although a little too small, lay peacefully between its four familiar walls. A collection of textile samples lay spread out on the table - Samsa was a travelling salesman - and above it there hung a picture that he had recently cut out of an illustrated magazine and housed in a nice, gilded frame. It showed a lady fitted out with a fur hat and fur boa who sat upright, raising a heavy fur muff that covered the whole of her lower arm towards the viewer. Gregor then turned to look out the window at the dull weather. Drops"
assert (unzip(zip(someLongTxt)) == someLongTxt)

Thursday, March 24, 2016

Playing with xml

XML is a convenient way to define external DSL. On one of my previous posts (Filtering using Scala Parser Combinators) I used parser combinators JavaTokenParsers with PackratParsers This time I will use different approach using XML.
XML is easy to handle by non-developers and easy to read. We can query XML using XPATH like expressions and using pattern matching. In this post we will define some business rules using XML and we will treat our XML as first class citizen. Source code can be found here.
let's define our DSL :
Our rules will use AND/OR operators but to make it more interesting, our operators will not use the following pattern:
<expression><operator><expression> (where expression evaluates to boolean).
because XML have opening and closing tags we can define something like :
<operator><expression1><expression2>...<expression N></operator>
now we can define our AND/OR operators :
  • AND: a set of expressions that all must be true
  • OR : a set of expressions that at least one must be true
  • SINGLE: contain our value and the operator used to evaluate the expression.
for example we want our condition to look something like this :
 val simpleRuleXml =
      <RULE id="1" description="5 top" status="true">
        <CONDITIONS>
          <OR>
            <SINGLE operator="eq"> 2 </SINGLE>
            <SINGLE operator="eq"> 4 </SINGLE>
            <SINGLE operator="eq"> 8 </SINGLE>
            <AND>
            <SINGLE operator="gt"> 25 </SINGLE>
            <SINGLE operator="st"> 100 </SINGLE>
            </AND>
          </OR>
        </CONDITIONS>
      </RULE>
that means that our value must be one of the following values: 2 or 4 or 8 or between 25 to 100
let's start by defining our value objects
sealed trait Expression

case class Or(ps: Seq[Expression]) extends Expression

case class And(ps: Seq[Expression]) extends Expression

case class Single(operand:String, operator:String) extends Expression
working with xml allows to navigate between the nodes and parse it very easily by using pattern matching and parse it recursively :
for example this expression will match all nodes between the TAG and bind it to the variable xs using a wild card _ and a Kleen star * which actually means "match any sequence " <TAG>{ xs @ _* }</TAG>
object Expression{
  def apply(ns: NodeSeq):Expression ={
    ns.head match  {
      case <CONDITIONS>{ xs @ _* }</CONDITIONS> => Expression(xs) 
      case <AND>{ xs @ _* }</AND> => And(xs map(Expression(_)))
      case <OR>{ xs @ _* }</OR> => Or(xs map(Expression(_)))
      case <SINGLE>{ s @ _* }</SINGLE> =>Single(s.text,ns\@"condition")
    }
  }
}
That's it ! well , almost... we still need to evaluate our rule, and check if our conditions are met .
object Rule{
  def apply(node: Seq[xml.Node]): Rule = {
    val id =node\s"@id"
    val desc =node\s"@description"
    val conditions = (node\\"CONDITIONS").map(Expression(_))
    Rule.apply(id.text,desc.text,conditions )
  }
}
A rule is mainly a set of conditions that needs to be met. As described above - all expressions between the AND tags must be true,
at least on expression between OR tags must be true.


case class Rule(id:String,desc:String,conditions:Seq[Expression]){
  def isValid[T:Ordering](value:T)(implicit f:String => T):Boolean = {
    def run (pr:Expression):Boolean = {
      pr match {
        case Single(name,operator) =>
          import scala.math.Ordering.Implicits._
            val n = f(name)
            operator match {
              case "eq" => value == n
              case "gt" => value > n
              case "st" => value < n
            }
        case And(ps) => ps forall run
        case Or(ps) => ps exists run
      }
    }
    conditions forall run
  }
}
and we are done !

for completion let's define some explicits so we can convert our value in the xml which is a String to T .
object Implicits {
//Since Java does not define toInt on String but on Int, so we need to be define which "toInt" implicit to use  
  implicit def string2Int(s: String): Int = augmentString(s).toInt
  implicit def string2Double(s: String): Double = augmentString(s).toDouble
  implicit def string2Float(s: String): Float = augmentString(s).toFloat
}

usage (using the above xml) :
val trimmedNode = scala.xml.Utility.trim(simpleRuleXml)

    val v = Rule(trimmedNode)
    assert(v.isValid(2))
    assert(v.isValid(4))
    assert(v.isValid(6))
    assert(!v.isValid(7))
    assert(v.isValid(30))
    assert(!v.isValid(900))
hope you enjoyed it . The Complete Source Code can be found here ,Feedback and remarks are always welcome