Monday 26 February 2024

MongoDB backup restore to remote locations

 

Mongo DB backup restore from one remote location to another remote location

  1. Login to a machine where mongoshell is installed.
  2. From inside mongoshell run the command: mongodump --uri "mongodb://:@/" --archive | mongorestore --uri "mongodb://:@/" --archive --nsExclude "admin.system.*"
  3. Reference: https://www.mongodb.com/docs/atlas/import/mongorestore/

Tuesday 21 December 2021

SiteMinder SSO Logout with Apache OIDC

To implement a Logout functionality with SiteMinder SSO OIDC, you need following configuration in your auth_openid.conf file:


<Location /logout >
    Redirect 302 "https://<siteminder-host>:<siteminder-port>/<sso-logout-URL>"
</Location>
      

Then in your application you will have to use the logout URL as:

<OIDCRedirectURI>?logout=https://<apache-host>:<apache-port>/logout
  

The above URL portion <OIDCRedirectURI>?logout= will cause the OIDC Session cached by Apache WS to get invalidated. Then it would redirect to /logout context configured in your Apache configuration. The /logout location configured in your Apache will cause a redirect to Avaya SSO logout URL which will ensure that SSO session used is invalidated in SSO Policy servers.

The following diagram shows a high level flow:



Friday 27 August 2021

Customize SSH Window Title in PuTTY Client for CLI Invocation

customize-window-title-in-PuTTY.md

In this blog, we will see how can we configure PuTTY to allow for customizable Window Title when launched from CLI.

When you open a PuTTY session using GUI the options is available under "Terminal" -> "Features" -> "Disable remote-controlled window title changing" where you can control Windows title change, and you can customize Window title under "Window" -> "Behaviour" -> "Window title:".

But in this blog, I will talk about how to permanently configure this option so that whenever you launch any PuTTY session using CLI this feature is available to you. The following steps you need to perform:

  • Step-1:

    • Since PuTTY keeps its configuration in Windows Registry, we will have to modify the Registry entry to change the default settings and select the option "Terminal" -> "Features" -> "Disable remote-controlled window title changing"

    • Run this command using Windows Command Prompt:

      • REG ADD HKEY_CURRENT_USER\Software\SimonTatham\PuTTY\Sessions\Default%20Settings /t REG_DWORD /v NoRemoteWinTitle /d 1 /f

    • This is a one-time activity only. If you upgrade or re-install PuTTY you will have to re-run the above command.

  • Step-2:

    • Now you can run a PuTTY session specifying the custom window title.

    • Example, run the command:

      • putty.exe -ssh root@10.133.90.12 -loghost "My Personal Title"

    • This will open the PuTTY session for the IP 10.133.90.12 using root user and the SSH Window title will be "My Personal Title".


Sunday 29 March 2020

RedHat EAP 7 Undertow Performance Tuning

RedHat EAP 7 Undertow Performance Tuning

Problem Statement

In our project we were using long polling http servlets using Netty framework. RedHat from EAP 6 TO EAP 7 has moved underlying web container from JBossWeb to Undertow framework. We faced issue in EAP 7 with default settings, where after some HTTP client requests the server was stopped responding to subsequent requests.

Enviornement

  • Oracle Linux 7.x 64-bit

  • 8 core CPU, 16 GM RAM

  • RedHat EAP 7.2 base version

  • Java Oracle 1.8

  • Max Client HTTP blocking requests - 5000

    • these are long polling requests being kept open for 24x7

Solution Description

The problem we found was, the default tuning parameters comes along with EAP 7 undertow.

Default Settings

  • The undertow subsystem relies on the io subsystem to provide XNIO workers. This option specifies the XNIO worker that the listener uses. By default, a listener uses the default worker in the io subsystem.

Example 1. EAP 7 Domain.xml (before tuning)
<subsystem xmlns="urn:jboss:domain:undertow:7.0" default-server="default-server" default-virtual-host="default-host" default-servlet-container="default" default-security-domain="other">
    <buffer-cache name="default"/>
 <server name="default-server">
  <http-listener name="default" socket-binding="http" redirect-socket="https" enable-http2="true"/>
  <https-listener name="https" socket-binding="https" ssl-context="spirit-server-ssl-context" enable-http2="true"/>
<subsystem xmlns="urn:jboss:domain:io:3.0">
 <worker name="default"/>
 <buffer-pool name="default"/>
</subsystem>
  • All listeners are tied to an XNIO Worker instance. Usually there will only be a single worker instance that is shared between listeners, however it is possible to create a new worker for each listener. The worker instance manages the listeners IO threads, and also the default blocking task thread pool.

    • IO Threads, perform non blocking tasks, default thread count is cpuCount * 2

    • Task Threads, perform blocking operations (such as Servlet requests), default thread count is cpuCound * 16

  • With 5000 HTTP requests load (long polling) we observed only 128 requests served correctly, remaining requests were stuck with no HTTP response coming to clients. This was due to default worker thread pool settings which in our case, 8 core CPU * 16, meaning 128 threads.

After Tuning

Example 2. EAP 7 Domain.xml (after tuning)
<subsystem xmlns="urn:jboss:domain:undertow:7.0" default-server="default-server" default-virtual-host="default-host" default-servlet-container="default" default-security-domain="other">
    <buffer-cache name="default"/>
 <server name="default-server">
  <http-listener name="default" max-connections="100000" socket-binding="http" redirect-socket="https" enable-http2="false" worker="http-worker"/>
  <https-listener name="https" max-connections="100000" socket-binding="https" ssl-context="spirit-server-ssl-context" enable-http2="false" worker="http-worker"/>
<subsystem xmlns="urn:jboss:domain:io:3.0">
 <worker name="default"/>
 <worker name="http-worker" task-core-threads="7000" task-max-threads="7000" task-keepalive="60000" />
 <buffer-pool name="default"/>
</subsystem>

Test Results

  • With 5000 HTTP requests load (long polling) and tuning we observed all 5000 requests served correctly. This was due to tuning where we made worker thread pool settings as 7000 core (first) and max (after) threads.

Saturday 11 May 2019

Tomcat Custom Valve to extract UserId out of HTTP SSO Headers and build X509 Certificate

Evernote Export

Overview
In this blog post i am going to show how to write a Tomcat Custom Valve. In my project I had a unique requirement of fetching user name/id from HTTP headers (being populated by a front Web Server responsible for authenticating user with SSO credentials) and then building X509 certificate programatically so that applications deployed having auth-method as CLIENT-CERT can pass.

Pre-requisite
  • Tomcat 9
  • Java 8 or above
  • A Front-end Web Server performing SSO authentication and then populating HTTP headers with SSO information OR you can simulate it via POSTMAN or other HTTP Requests sending tools.
    • For example in our case the http headers "sm_user" and "sm_session" was populated by userid information.

Steps to follow
  1. Create a Java Project in Eclipse IDE.
  2. Create a Java Class as below for a Custom Valve, remember you will have to import jars from <Tomcat-Home>/bin and <Tomcat-Home/lib folders:
import java.io.IOException;
import java.math.BigInteger;
import java.security.GeneralSecurityException;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.PrivateKey;
import java.security.SecureRandom;
import java.security.cert.Certificate;
import java.security.cert.X509Certificate;
import java.util.Date;
import javax.servlet.ServletException;
//import jars from <TOMCAT-HOME>/lib
import org.apache.catalina.Globals;
import org.apache.catalina.connector.Request;
import org.apache.catalina.connector.Response;
import org.apache.catalina.valves.ValveBase;
//import jars from <TOMCAT-HOME>/bin
import org.apache.juli.logging.Log;
import org.apache.juli.logging.LogFactory;
import sun.security.x509.AlgorithmId;
import sun.security.x509.CertificateAlgorithmId;
import sun.security.x509.CertificateSerialNumber;
import sun.security.x509.CertificateValidity;
import sun.security.x509.CertificateVersion;
import sun.security.x509.CertificateX509Key;
import sun.security.x509.X500Name;
import sun.security.x509.X509CertImpl;
import sun.security.x509.X509CertInfo;
public class X509CertCreatorCustomValve extends ValveBase {
private static final Log log = LogFactory.getLog(X509CertCreatorCustomValve.class);
@Override
public void invoke(Request request, Response response) throws IOException, ServletException {
log.info("Custom valve X509CertCreatorCustomValve is called");
String userid = request.getHeader("sm_user");
if (userid == null || userid.isEmpty()) {
userid = request.getHeader("sm_universalid");
}
log.info("UserId extracted from SSO Headers: " + userid);
if (userid != null) {
try {
X509Certificate certs[] = (X509Certificate[]) request.getAttribute(Globals.CERTIFICATES_ATTR);
if ((certs == null) || (certs.length < 1)) {
KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("RSA");
KeyPair keyPair = keyPairGenerator.generateKeyPair();
String distinguishedName = "CN=" + userid + ", OU=TSD, O=XUZ, L=US1, S=NJ, C=US";
Certificate certificate = generateCertificate(distinguishedName, keyPair, 365, "SHA256withRSA");
certs = new X509Certificate[1];
certs[0] = (X509Certificate) certificate;
request.setAttribute(Globals.CERTIFICATES_ATTR, certs);
}
} catch (Exception e) {
log.error("Error preparing X509 certificate", e);
}
}
getNext().invoke(request, response);
}
private X509Certificate generateCertificate(String dn, KeyPair pair, int days, String algorithm)
throws GeneralSecurityException, IOException {
PrivateKey privkey = pair.getPrivate();
X509CertInfo info = new X509CertInfo();
Date from = new Date();
Date to = new Date(from.getTime() + days * 86400000l);
CertificateValidity interval = new CertificateValidity(from, to);
BigInteger sn = new BigInteger(64, new SecureRandom());
X500Name owner = new X500Name(dn);
info.set(X509CertInfo.VALIDITY, interval);
info.set(X509CertInfo.SERIAL_NUMBER, new CertificateSerialNumber(sn));
info.set(X509CertInfo.SUBJECT, owner);
info.set(X509CertInfo.ISSUER, owner);
info.set(X509CertInfo.KEY, new CertificateX509Key(pair.getPublic()));
info.set(X509CertInfo.VERSION, new CertificateVersion(CertificateVersion.V3));
AlgorithmId algo = new AlgorithmId(AlgorithmId.md5WithRSAEncryption_oid);
info.set(X509CertInfo.ALGORITHM_ID, new CertificateAlgorithmId(algo));
// Sign the cert to identify the algorithm that's used.
X509CertImpl cert = new X509CertImpl(info);
cert.sign(privkey, algorithm);
// Update the algorithm, and resign.
algo = (AlgorithmId) cert.get(X509CertImpl.SIG_ALG);
info.set(CertificateAlgorithmId.NAME + "." + CertificateAlgorithmId.ALGORITHM, algo);
cert = new X509CertImpl(info);
cert.sign(privkey, algorithm);
return cert;
}
}
  1. Export the class as a jar by using Eclipse jar export feature or build it manually from the compiled class.
  2. Put the jar in <TOMCAT-HOME>/lib folder
  3. Add a <valve> entry in <TOMCAT-HOME>/conf/server.xml file under <Host> tag as mentioned below:
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true">
<Valve className="X509CertCreatorCustomValve"/>
........
........
</Host>
  1. Start the Tomcat and test your application. Remember the pre-requisite setup you need, your web application must have CLIENT-CERT auth methnd configured in web.xml and http header needs to be populated before hitting your application.


Thank You!

Tuesday 5 March 2019

Running Linux CLI commands in Windows 10

Objective


Enable running Linux CLI native commands in Windows 10 machine.
FYI, it's Windows 10 provided feature, more information here: https://docs.microsoft.com/en-us/windows/wsl/faq 

Steps

  • Enable Linux bash in Windows 10:
    • Go to "Windows features on or off" dialog box:
    • Turn on the "Windows Subsystem for Linux" option:
  • Run bash and install Linux distribution:
    • Open Windows "CMD" command prompt and type "bash" command
    • if you get a message saying "Linux Distributions" not available, the run command "lxrun /install" to install Linux Distribution
  • Test
    • Once you get the bash command prompt, you can type Linux commands now.


Thank You!