How To Wait Until A Glue Crawler Is Complete In A Step Function

Posted on June 15, 2022 by Michael

In a Step Function we’re able to wait until a Glue Job is complete before moving on to the next step by checking “Wait for task to complete”. Tragically, not so with Glue Crawlers 🙁

There are several ways to wait for a Crawler to complete. The easiest is to poll the state of the Crawler using a Glue:GetCrawler step. If the output is READY then move on, otherwise wait for a period and check again.

Remember to add the Output from the Glue:GetCrawler step. This allows us to check it in the Choice State.

In the Choice State test that the $testCrawlerState.Crawler.State is equal to the string constant READY.

 

Posted in Programming | Tagged , , , , | Leave a comment

Almost Useless Top Tip for Today: Lombok @RequiredArgsConstructor

Posted on July 9, 2021 by Michael

Wiring up Spring Components using @Autowired is convenient but makes testing harder. A better solution is passing dependencies in through a constructor. However, that means creating a constructor. Lombok to the rescue with the @AllArgsConstructor annotation which will create a constructor that requires all your Class variables. Great for most situations but what if you want only some variables initialised through the constructor? Use the @RequiredArgsConstructor and make your variables final.


@Component
@RequiredArgsConstructor
public class MyService {
private final MyFirstDependency firstDependency;
private final MySecondDependency secondDependency;

private MyThirdDependency optionalDependency;
}

Posted in Programming | Tagged , , | Leave a comment

Spring Slueth Stops JMSListener Consuming Messages

Posted on October 26, 2019 by Michael

Adding the following dependency causes JMS messages to stop being consumed.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-sleuth</artifactId>
    <version>2.1.0.RELEASE</version>
</dependency>

The issue is that Spring Sleuth prevents the JMSListener from being registered as per:
Jms listeners not running

To fix use version 2.1.1.RELEASE of spring-cloud-starter-sleuth:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-sleuth</artifactId>
    <version>2.1.1.RELEASE</version>
</dependency>
Posted in Programming | Tagged , , | Leave a comment

Posted on April 11, 2017 by Michael

On Ubuntu 16.04 running pharo, from the Moose Technology download, results in the following error:

unknown option: -plugins

To fix, change:

# execute
exec "$LINUX/pharo" \
-plugins "$LINUX" \
-encoding utf8 \
-vm-display-X11 \
"$image"

To:

# execute
exec "$LINUX/pharo" \
--plugins "$LINUX" \
--encoding utf8 \
"$image"
Posted in Programming | Tagged , , | Leave a comment

Error on ‘inverse_transform’ of LabelEncoder

Posted on January 9, 2017 by Michael

If you get the following error when calling inverse_transform on sklearn.preprocessing.LabelEncoder

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

You could be passing in a numpy.ndarray of strings, not numeric values.

Here’s a snippet to encode and decode a column from the Titanic training set:

from sklearn.preprocessing import LabelEncoder

encoder = LabelEncoder()
sex_cat = train_df["Sex"]
sex_cat_encoded = encoder.fit_transform(sex_cat)
print(encoder.inverse_transform(sex_cat_encoded))
# print(encoder.inverse_transform(sex_cat)) This will give you the error

Posted in Programming | Tagged , , | Leave a comment

October Jozi JUG – [Super Awesome Speaker] Nitsan Wakart presents JVM Profiler Mechanics

Posted on October 6, 2015 by Michael

Don’t miss this one! Nitsan will take us through all manner of dark JVM arts:

A blogger and a coder with a pedantic passion for performance. My work has spanned army intelligence systems, dot.com era startups, financial institutions and innovative product companies. Currently the Lead Performance Eng. for Azul Systems by day, and main contributor to kickass lock free data structures library JCTools by night.

We will take a deep dive into the guts of Honest-Profiler, an unbiased sampling CPU profiler for the JVM, and the JVM internals which enable it to work.

• Introduction: CPU profiling, sampling and biases explained.

• How does it hang together? interrupts, non-blocking code and an unofficial API

• Sampling the instruction pointer: how it works and limits of accuracy

• Walking the stack: demangling a mixed Java/native stack, challenges and limitations

• Reading a profile: flat profile vs. tree profile vs. flamegraphs

The talk will include some counter examples from other profilers(JVisualVM/JMC/YourKit/ZVision).

See you there!

Posted in Uncategorized | Leave a comment

Grails Bootstrap Form Field Error Template

Posted on October 4, 2014 by Michael

Define a Grails template in /layouts/_formFieldError.gsp
<div class="form-group has-error">
<g:hasErrors bean="${fieldModel}" field="${fieldName}">
<div class="errors">
<g:eachError bean="${fieldModel}" field="${fieldName}">
<span class="help-block"><g:message error="${it}"/></span>
</g:eachError>
</div>
</g:hasErrors>
</div>

Usage
<g:render template="/layouts/formFieldError" model="[fieldModel: myModelObject, fieldName: 'myFieldName']" />

Posted in Programming | Tagged , , | Leave a comment

Making Sure Our Technical Communities Are Safe Spaces For All

Posted on September 8, 2014 by Michael

The number of technical user groups in South Africa has grown and we’ve now got several vibrant communities. Which is awesome!

If we want to make sure our communities continue to be as awesome and welcoming as they are, we need to take steps to ensure our events and on-line spaces stay safe. The easiest of which is the implementation and enforcement of an anti-harassment policy.

How easy is it? You can lift, almost word for word, a policy from the Geek Feminism Wiki. We’ve done so at the Jozi JUG.

If your group hasn’t instituted an anti-harassment policy, email the organizers and let them know they need one and where to get it!

Duke gives the thumbs up to safe communities

Duke gives the thumbs up to safe communities

Posted in Event, Programming | Leave a comment

August Jozi JUG – Docker and MongoDB

Posted on August 19, 2014 by Michael

Want to learn all about Docker? Then come to the August Jozi JUG! RSVP and details on Meetup.

Docker

Docker

New to Java and keen on learning about MongoDB? The Next Gen talk is just for you! RSVP and details on Meetup.

MongoDB

MongoDB

 

Posted in Event | Tagged , | Leave a comment

June Jozi JUG – Javascript, Nashorn and Project Avatar

Posted on June 18, 2014 by Michael

The June Jozi JUG will take place on 30 June at BBD from 18:00. So come through for great presentations as well as free food and drink. To RSVP please go to the groups Meetup page.

Next Gen : The Jozi JUG Code Dojo Second Kata

The Jozi JUG code dojo’s second white belt kata will be explained.

The Dojo is a totally free place where new Java developers in South Africa can learn modern techniques of the Java ecosystem.

The kata includes explaining the code of the first kata, modifying the program and pushing the code to github.

Come on over!

Come on over!

History of JavaScript and new exciting features of Nashorn and Project Avatar

JavaScript is all the rage! If you have not heard of node-this-node-that there where have you been?

Not to want to miss all the attention, Oracle has put its brightest minds in upgrading Java’s JavaScript engines with 95% Java-To-Node compatibility.

Join Rory as we discover the history of JavaScript and the new exciting features of Nashorn and “Project Avatar”

See you there!

Posted in Uncategorized | Leave a comment