Magento 2 Disable advanced search link in topmenu

If you’re going to override the Magento theme then I really would advise against modifying  the Magento core files. A better practice is to create your own theme and override all the files you need. This way you can update Magento with much less worries. How you create your own theme is for another episode.

Supposing you have created your own theme you should have a directory structure that looks something like this:

In my case, the theme is called desktop.

In the desktop folder, create a directory called Magento_Search.

And in the Magento_Search folder, create another directory called layout

Now, with nano, create a file called default.xml

So if you were paying attention you now have:

In the default.xml file, insert:

Clear the cache and you’re good to go.

Saving ESCAM QF001 IP camera RTSP stream to disk

The ESCAM QF001 is a very cheap (but fairly decent) IP camera. Recently I bought three from AliExpress and payed about € 30 a piece. After testing I’m planning to buy three more. The only complaint I have is that they make kind of a hissing sound but the video and audio quality is pretty good for it’s price. Here is a review.

The goal is to save all the video / audio streams from the different camera’s to a hard disk so it can be viewed later. I’m having a Raspberry Pi Model B+ as a fileserver.

So in my case, I’m using six camera’s but this works as good for 1 as it does for 6 or for 10. Let’s get started.

You’ll need:

  • IP camera(‘s) that support(s) an RTSP stream;
  • a Linux fileserver. I’m using a Raspberry PI with Raspbian on it;

If you want to configure these camera’s, you’ll need VMS or Video Management Software. It’s probably developed in China but it’s not too bad, once you get to know it.

  • First I’m going to make sure the camera has a static IP so I can always reach it on the same address. This also will make things easy for our script. You select your camera (left treeview), click Device config and click on Network.

  • Second, I’m going to setup a password for the Administrator and make a separate account for the user that’s going to login to the camera and will fetch the RTSP stream. Same as before: select your camera (in the left treeview), click on Device config and choose Tool manager.
  • In the tab User manager I have added a user called ipcamera. You can remove rights from the user but I have not really bothered.

 

For now, we are done with the camera’s so we’ll get started on the Raspberry PI fileserver. I’m having an external hard disk attached to it with the following directories:

 

This is set up so the different camera’s can stream into their own directory.  I’m also making a user called ipcamera on the Raspberry Pi. This one will be used exclusively for streaming and you could tune the access this user has so it can just write into the different CAM folders.

I’m using avconv to convert the h264 RTSP stream into something VLC can read. The avconv package is part of winff, so I’m installing that on the PI.

This long stream will be split up into parts of 10 minutes. We’ll need to kill this avconv process and for that we’ll need pkill.

Logged in as the ipcamera user, I’m making two files in my homedirectory: /home/ipcamera:

  • recordIpCameras.sh

This file saves the live RTSP feed from the different camera’s. Don’t forget to change the user / password and chmod +x.

  • cleanIpCamera.sh

The script above generates the video files for us. This one deletes them after a set period of time.

In conclusion we’re going to add these two scripts to the crontab of the ipcamera user.

  • Login as the ipcamera user

Add these two lines to your crontab file:

Reboot (for the fun of it), wait until a tenth minute (like: 7:10, 7:20, 7:30) and watch the feeds pour in.

Creating a new Magento 2 Integration to consume with c-sharp and the SOAP API

So you want to use c-sharp to access your Magento 2 instance? Join the club. Unfortunately, there’s not a lot of documentation so I’m writing stuff on my blog as I go along, hoping it will be useful for someone.

If you want to access your Magento 2 instance with c-sharp then for starters, you’ll need to create an Integration.

In Magento 2 goto

  • System, Extensions -> Integrations

Configure as follows:

 

  • In the API tab you can configure the access of the user:

Magento 2 SOAP API with C Sharp adding Service References

If you’re just starting out with Magento 2 and you’re coming from Magento 1, you’ll immediately see the difference in terms of the SOAP API.

It seems the developers have opted to split the functionality into several smaller units. It increases security but adds a little bit of complexity. Nothing that can’t be overcome.

You can combine the service request using commas so you don’t need to have 50 or so different services. You can read all about it on the Magento SOAP reference page.

I’ve grouped them into the Magento categories on their website. You can see the Service Reference URL for Visual Studio below together with the ServiceReference name. Don’t forget to change www.domain.com to your domain.

Warning: i’ve had mixed results with this approach. Sometimes you’ll get messages like “catalogProductRepositoryV1SaveRequest not present”. Use it at your own risk but I would not recommend it. Splitting the services up solves the issue…

ServiceReferenceMagentoBackend

ServiceReferenceMagentoBundle

ServiceReferenceMagentoCatalog

* For some reason, the service catalogProductGroupPriceManagementV1 cannot be added so I have omitted it.

ServiceReferenceMagentoCheckout

ServiceReferenceMagentoConfigurable

ServiceReferenceMagentoCustomer

ServiceReferenceMagentoDownloadable

ServiceReferenceMagentoEav

ServiceReferenceMagentoGiftMessage

ServiceReferenceMagentoIntegration

ServiceReferenceMagentoQuote

ServiceReferenceMagentoSales

ServiceReferenceMagentoTax

  • So now we import the individual service’s into Visual Studio like this:

  • And then we rinse and repeat for every service reference:

Magento 2 CatalogProductRepositoryV1SaveRequest not available in API

When you’re starting out with the Magento 2 API in c# (c sharp) you’ll notice it’s quite different from the previous version.

The API has been split up into different pieces to, among other things, address security concerns. This means you’ll probably need to add several Service References to get the job done.

My goal was to insert products trough the Web API. To my surprise it seemed the CatalogProductRepositoryV1SaveRequest method was missing from the http://www.xxx.yyy/soap/default?wsdl=1&services=catalogProductRepositoryV1 service…

I have not found any other solution than to override security in the file /vendor/magento/module-webapi/Model/AbstractSchemaGenerator.php.

Allowing anonymous web access does not help…

Now before you start making changes, it’s best to backup the file. Also note that this should be a temporary solution and it creates a security issue. Anyone can now see (but not use) this method when requesting the wsdl from the service.  Also, when updating Magento there is a probability it will be overwritten. You have been warned.

The two changes are highlighted in yellow.

 

Magento 2 sending e-mail trough Outlook 365 Smtp

Like many others out there, we’re running our own VPS (Virtual Private Server) for our Magento 2 instance. This allows us full control over the code, security and it’s also pretty fast.

When using a VPS, you’ll need to setup e-mail yourself. You can save yourself the trouble if you have an Office 365 subscription. The only thing you have to do then is to make Magento send e-mails using the Outlook 365 Smtp server.

  • Navigate to your Magento 2 app/code directory

  • Make a directory structure like this:

 

The text in red is what’s probably different in your instance or what you’ll have to change.

  • In the folder Smtp, make a file called registration.php and insert the content from below. Make sure you change Organization_Smtp to what is appropriate for you. Don’t forget to chmod +x it :-).

  • In the folder etc, you’ll need to make two files: di.xml and module.xml

Insert the content below for di.xml and don’t forget to change Organization to what is appropriate for you.

Insert the content below for module.xml and don’t forget to change Organization to what is appropriate for you.

  • In the folder Module, you’ll need to make a file called Transport.php and insert the content below. Change Organization and username / password to what’s appropriate for you. Don’t forget to chmod +x it.

  • Now if you would order something, the confirmation e-mail will be sent trough the Outlook 365 Smtp service (if the sender is a valid Office 365 client).

Machine Translation using Azure and Universal Windows App

I’ve been using Microsoft’s machine translations to translate the description of about 2000 products from Dutch to French, German and English. The results are fair so no complaints there.

Recently I got an e-mail from Microsoft stating the following:

Action recommended: Migrate Microsoft Translator API to Azure—limited subscription access in Azure DataMarket through April 30, 2017

So, if we need to port code, why not port it to the Universal Windows App platform? Turns out it’s a fairly simple process:

  • Created a class called AzureSOAPClient

  • All one has to do now is to instantiate AzureSOAPClient using the subscription key and call the async method TranslateAsync.

You can find the GitHub repository here.

Controlling an Arduino with a WiFi ESP8266 adapter using a Windows 10 Universal App

If you’ve followed my previous post ‘Controlling an Arduino with a Bluetooth HC-05 adapter using a Universal App‘ you’ll notice many similarities when reading the instructions below.

There are, however, some advantages of using a WiFi ESP8266 module instead of a Bluetooth HC05 module:

  • the wireless range is larger;
  • the amount of devices that use WiFi is greater.

The goal is to communicate using WiFi from a phone, computer, … with an Arduino equipped with a ESP8266 module and drive a LED. A simple and fun proof of concept.

What do you need?

  • some code in C# for the Windows 10 Universal App platform;
  • an Arduino. I’m using a Leonardo;
  • a cheap ESP8266-01 module and a FTDI cable.
Part 1: The Aduino / ESP8266-01

There are many tutorials out there that show you how to connect your Arduino to an ESP8266. For example: this one or this one so you’ll forgive me if I’m not going to explain how to do that. The basic schematic however is this:

Ideally you could place a voltage divider between the Arduino Tx and ESP8266-01 Rx so the input pin of the ESP8266 is not subjected to 5V.

You could also place a LED on the Breadboard somewhere and attach it to your Arduino. We’re going to use the built in LED on pin 13.

You’ll notice that we have connected the Arduino’s Tx to the ESP8266-01’s Rx and the Arduino’s Rx to the ESP8266-01’s Tx and we didn’t even bother to use the ESP8266-01’s GPIO. That’s because we will relay all the messages sent via TCP/IP from our computer to the ESP8266 via Serial to the Arduino.

First, connect your ESP8266-01 via the FTDI cable to your computer. Make sure it’s in bootloader mode.

Then you’ll need to configure your Arduino IDE to compile for the ESP8266-01 (and not for Uno or Leonardo or …).

You can check out the code below as a reference. It doesn’t do much except it connects your ESP8266-01 to your SSID and retries if the network was not available.

The ESP8266-01 then waits for someone to connect to it and sends it a message. You’ll notice we don’t use the GPIO pins from the ESP8266-01. Instead we just forward the message trough it’s serial pins to our Arduino.

Program for the ESP8266-01

Program for the Arduino:

You could try to connect to your ESP8266-01 with a program like Putty and make sure everything is OK. Sending the message LED ON should light up your LED and LED OFF should dim it.

Part 2: The Universal App for Windows 10

Now, all the above… It’s not the main focus of today. I want to show you how easy it is to use the power of the Windows 10 Universal App API to connect to an ESP8266 module (or any other WiFi client).

The class TCPClient is kind of a wrapper around StreamSocket. It connects to a host and provides us with Events we can consume in the GUI. You could even recycle most of it if you would like to use Bluetooth.

The consumer (our GUI) is really basic and just allows us to connect and send a message over TCP/IP using an IP adress and a port.

If you start the application you’ll notice you can connect and there are two buttons. Clicking LED ON should drive the LED high and clicking LED OFF should drive the LED low.

You can find the complete GITHub repository here.

In conclusion

I must admit I kind of have a love / hate relationship with those tiny ESP8266-01 modules…

They are very small and cheap but they have some disadvantages that you should be aware of.

ESP8266-01’s only have 2 GPIO pin’s but they need to be in a certain state when the ESP8266-01 is powered on or else you get into the bootloader mode. This makes your I/O more difficult. You could use ESP8266-12 instead.

If you want to use an Arduino to expand the IO pins then know that these modules require 3.3V. If you connect them to the 3.3V pin, the Arduino can barely supply enough power. Depending on what you’re planning to drive, the 3.3V supply from the Arduino is not enough and you should look for an alternative solution (like a voltage divider on the 5V pins or a 3.3V voltage regulator or an external power supply).

With all that being said I must tell you that I grew out of using the ESP8266-01. They are too much of a hassle for me. I’m now in the habit of going nuclear and just use a Raspberry PI 2 / 3 with Windows 10 IOT. It’s just easy to be able to remote debug over TCP/IP, have a GUI, NuGet packages, … They are worth the extra price for me.

If you need to have exact timing (certain sensors, a LED strip), you’ll need a microcontroller like the Arduino. I would suggest to just get the Arduino Uno WiFi.

Controlling an Arduino with a Bluetooth HC05 adapter using a Universal App

Today we’re going to combine three of my favorite things in the world:

  • some code in C# for the Windows 10 Universal App platform;
  • an Arduino;
  • cheap wireless transmission like the Bluetooth HC-05 module.

The goal is to communicate using Bluetooth from a phone, computer, … with an Arduino equipped with a HC-05 module and drive a LED. A simple and fun proof of concept.

The HC05 is a cheap Bluetooth module that you can use to extend the wireless capacity of your Arduino. I’m not going to go into the basics of this module as there’s an excellent ‘how-to’ available on this page.

You can get a HC-05 module from sites like AliExpress or Banggood for 3$. If you want it faster or want to support your country’s economy you can probably buy it at a local reseller.

The HC-05 has 6 pins but only four of them really matter for our purposes of today: Vcc (typically 5V), GND (0V), Txd (transmit) and Rxd (receive).

It seems quite obvious that we’re going to communicate via the serial protocol from our Arduino to this module and back.

Connect your Arduino’s Tx to the HC-05’s Rx and connect your Arduino’s Rx to the HC-05’s Tx. Yes, it needs to be crossed.

Connect a LED to an output pin of your choice. Note that I have used an Arduino Leonardo instead of an Uno. Probably should not make too much difference.

We do not need to worry about the power supply. The HC-05 is 5V tolerant and the Arduino can supply more than enough power to the module since we’re only driving a LED.

Load this code up into your Arduino:

You could already go ahead and test this out by opening your Serial monitor. If you type in LED ON, the LED should start to light up.

The second part of this project is to use a computer (or smartphone) to drive this LED. We’re using the Windows 10 Univeral App platform so we’re striking two birds with one stone. Offcourse you’ll need a computer with a Bluetooth connection or a smartphone running Windows 10 Mobile.

I’ve made a class BluetoothClient that’s a wrapper for StreamSocket. It uses the Singleton design pattern so a consumer can only create one of them. For our conveniance, events are created so we are notified when something happens.

Only thing left now is a GUI to consume the class. As you’ll notice it’s pretty straightforward.

Don’t forget to add Bluetooth capabilities in the application manifest file.

The only thing now is to start up the application and fill in the MAC address of your HC-05 module. You can leave Address on 1. If you connect and click on LED On, you should be seeing your LED in action on the Arduino.

 

You can find the GIT repository here. Have fun!

Making a jailed user that’s only able to log-in in Ubuntu

Recently I needed to give someone read-only access to a production database that’s sitting on top a Virtual Private Server (VPS).

Since the database only allows connections from the localhost, you need to establish a SSH tunnel to the VPS using Putty. In order for the tunnel to work, the user has to be logged in. This means a shell has to be opened…

No way I was going to give that special someone access to the file system and no way I was opening up the database so you could connect to it from anywhere. I needed a shell that could only log in, nothing more.

Since I’m kind of a novice to Linux my first guess was to use rbash but after doing some reading (yes, reading… That’s a (Cinama) sin, right?) I found out this was like closing all the doors but with the key still in the lock.

This is what rbash restricts:

  • cd command (Change Directory)
  • PATH (setting/ unsetting)
  • ENV aka BASH_ENV (Environment Setting/ unsetting)
  • Importing Function
  • Specifying file name containing argument ‘/’
  • Specifying file name containing argument ‘-‘
  • Redirecting output using ‘>’, ‘>>’, ‘>|’, ‘<>’, ‘>&’, ‘&>’
  • turning off restriction using ‘set +r’ or ‘set +o’

However, I noticed you could still use vi or nano to read files if you’d knew the exact directory path. Not a risk you’re willing to take.

It seemed there was only one option and that was to jail the user to a chroot environment. Sounds difficult, doesn’t it?

Turns out it wasn’t.

All you need is a script called jailkit

First you’ll need to install it. It’s only available from source, no .deb packages.

I made a jail directory in the home directory and adjusted the permissions so only root has access.

Next you’ll have to do some configuring. The user I would like to have limited access for is called Mark (with a K).

Now we check the passwd file and modify it if necessary (maybe you’re more of a Zsh person. If the user needs to log into the shell using SSH, the shell should be able to use SSH connections. I’ve tried to use rbash there but that did not work.

Be very careful that you config the /etc/passwd file in /home/jail. I’ve lost an hour or so because I tried to edit /etc/passwd and not /home/jail/etc/passwd.

Change it so it looks something like:

And now, the final step, we add basic shell utilities to the jail.

Check the connection with Putty, the user should be able to log in and has nothing more than a shell. Sure he or she can change directories or do stuff but only in his jail, not in the main filesystem.